Report  No.  cG-D-12-95 


Human  Factors  Evaluation  of  Electronic  Chart 
Display  and  Information  Systems  (ECDIS) 


U.S.  Coast  Guard 
Research  and  Development  Center 
1082  Shennecossett  Road 
Groton,  CT  06340-6096 


and 


MarineSafety 

international 


Computer  Aided  Operations  Research  Facility  (CAORF) 
National  Maritime  Research  Center 


Kings  Point,  NY  11024 


FINAL  REPORT 
FEBRUARY  1995 


This  document  is  available  to  the  U.S.  public  through  the 
National  Technical  Information  Service,  Springfield,  Virginia  22161 


This  docnuaeat  hoi  b©«t»  approved 
for  public  release  and  solej  its 
distribution  »  unlimited _ ' 


Prepared  for; 


U.S.  Department  of  Transportation 

United  States  Coast  Guard 

Office  of  Engineering,  Logistics,  and  Development 

Washington,  DC  20593-0001 


19950626  066 


NOTICE 


This  document  is  disseminated  under  the  sponsorship  of  the 
Department  of  Transportation  in  the  interest  of  information 
exchange.  The  United  States  Government  assumes  no  liability 
for  its  contents  or  use  thereof. 


The  United  States  Government  does  not  endorse  products  or 
manufacturers.  Trade  or  manufacturers'  names  appear  herein 
solely  because  they  are  considered  essential  to  the  object  of 
this  report. 

The  contents  of  this  report  reflect  the  views  of  the  Coast  Guard 
Research  &  Development  Center.  This  report  does  not  consti¬ 
tute  a  standard,  specification,  or  regulation. 


/*/ 


W.  E.  Colburn,  Jr. 

Technical  Director,  Acting 
United  States  Coast  Guard 
Research  &  Development  Center 
1082  Shennecossett  Road 
Groton,  CT  06340-6096 


1 .  Report  No. 


2.  Government  Accession  No. 


Technical  Report  Documentation  Page 


3.  Recipient's  Catalog  No. 


CG-D-12-95 


4.  Title  and  Subtitle 

Human  Factors  Evaluation  of  Electronic  Chart  Display 
and  Information  Systems  (ECDIS) 


7.  Author(s)  M.W.  Smith,  R.A.  Akerstrom-Hoffman,  C.M.  Pizzariello, 

_ S.l.  Siegel,  T.E.  Schreiber,  and  I.M.  Gonin  _ _ 

9.  Performing  Organization  Name  and  Address 

U.S.  Coast  Guard  Computer  Aided  Operations  Research 

Research  and  Development  Center  Facility  (CAORF) 

1082  Shennecossett  Road  National  Maritime  Research  Center 

Groton,  Connecticut  06340-6096  Kings  Point,  New  York  11024 


5.  Report  Date 

February  1995  _ _ 


6.  Performing  Organization  Code 


8.  Performing  Organizafion  Report  No. 
R&DC  10/93 

MSI/CAORF  26-9038-01 A 


10.  Work  Unit  No.  (TRAIS) 


1 1 .  Contract  or  Grant  No. 

DTMA91-88-C-80024 


13.  Type  of  Report  and  Period  Covered 
I  Final  Report 


12.  Sponsoring  Agency  Name  and  Address  hinai  Mepon 

Department  of  Transportation  - - : — - - - 

U.S  Coast  Guard  14.  Sponsoring  Agency  Code 

Office  of  Engineering,  Logistics, 
and  Development 

Washington,  D.C.  20593-0001  _ _ _ _ 

15.  Supplementary  Notes  Coast  Guard  R&D  Center  Contacts:  Dr.  Myriam  Witkin  Smith,  SAB,  (203)  441-2844 

Project  2720,  Integrated  Navigation  Systems,  Dr.  Lee  Alexander  (203)  441-2639 

16.  Abstract 

This  report  describes  a  study  done  to  examine  the  contributions  that  ECDIS  might  rnake  to  the 
operational  practices  on  the  commercial  bridge.  Issues  examined  were:  the  contribution  to  the 
safety  of  navigation,  the  effect  on  the  navigational  workload,  the  features  required  during  route 
monitoring,  and  the  potential  contribution  of  integration  with  radar. 

Two  commercially-avaiiabie  ECDiS  devices  were  installed  on  the  simulator  bridge  at  MSI/CAORF. 
Expert  mariners  made  repeated  port  arrivals  and  departures.  A  variety  of  ship  and  mariner 
performance  measures  were  collected  and  extensive  debriefings  were  conducted. 

ECDIS  demonstrated  the  potential  to  increase  safety,  primarily  by  decreasing  the  cross  track 
distance  from  a  planned  track,  and  the  potential  to  decrease  the  workload  of  route  monitoring, 
primariiy  by  replacing  time-consuming  plotting  on  the  paper  chart.  For  route  monitoring, 
mariners  required  only  a  simple  display  outlining  safe  water,  but  recommended  access  to  a 
iarger  set  of  chart  features  as  reference.  Radar  integration,  as  implemented  on  the  devices  used, 
did  not  provide  the  ARPA  information  mariners  required. 


1 7.  Key  Words 

Electronic  Chart  Display  and  Information  Systems 
(ECDIS),  navigational  displays.  Integrated  Navigation 
Systems,  shiphandling  simultors,  operator  workload, 
performance  standards 


19.  Security  Classif.  (of  this  report) 


1 8.  Distribution  Statement 

Document  is  avaiiabie  to  the  U.S.  public  through 
the  National  Technical  Information  Service, 
Springfield,  Virginia  22161 

22.  Price 


UNCLASSIFIED 


Form  DOT  F  1700.7  (8/72) 


20.  SECURITY  CLASSIF.  (of  this  page) 

21.No.  of  Pages 

UNCLASSIFIED 

Reproduction  of  form  and  completed  page  is  authorized 


iii 


CO 

CL 

o 

I- 

o 

< 

LL 

z 

g 

CO 

CL 

LU 


O 

O 

g 

CL 

F" 

LU 


CO 

CD 

k. 

D 

CO 

CCS 

(D 


O 

CD 

2 

E 

o 


CO 

c 

_o 

'co 

u. 

o 

> 

c 

o 

o 

CD 

•4-» 

CO 

E 

x 

o 

L. 

a 

a 

< 


£Z  \ZZ 


o 

n 

E 

>> 

CO 


T3 

C 

L 

,o 


CD 

.9- 

Z3 

:e 


o 

c 

:: 

c 

0 

x: 

5 


o 

n 

E 

> 

CO 

IZ 


.E.£r  >E 


W  yj 

s 

11  111 


q^t  (O’-.  <0 
do  cri o 


“i51 


w 

0)  w 


5=  o  a  O’  o)  >s 


.E  >  E 

<U  (D 
^  ^  ^ 

2  to  0) 

3  3  3  i- 
CT  Cr  CT  O 
«  eo  to  C3 


(O 

^  q  ^  q 
d  T-  d  cvi 


CO 


0) 


•i:  0  o  £ 

s 

o  E  E  ^ 


CO  CO 
3  3 

a  cj 
to  to 


to  O 

l§ 

£  O 

o  C 

'3  (0 
0) 

I-  CO 
CO 

3  O 
CJ  0) 

to  £ 


CO 

CO 

< 


to  to  i: 
cu  *o  _ 
o  c  tr 
c  3  o 

3  O  £ 

o  a  to 


lO 

CO 

q  w  r- 
d  cvj 


^  *0 
to  *2  ^ 

^  ^  o  .y  .E 

•5  &  i  1  ■§  •§ 

5=  O  a  CJ  C3  o  o 


3 

o  <0 

-D  «  ^  ^ 


O 

k- 

3 

0  O 

-c  a 
"  E 
o 


o 


LU 

D 

-J 

O 

> 


11  ee. 


02 


6L 


8L 


ZL 


9L 


9L 


C\J 

:  £  CO 
-  ^  JZ 


o 

o 

o 

to  ▼“ 
£ 

CO  to 

W  ^  <u 
E  O)  c 

03  O  C 
CT  lac:  — 


o> 


in 

CO  CJ  <D  <D 

q  -r-  q  CJ 
6  d 


CM  o 


CO 

in 

CO 


to 

to  to  to  to 


to  <0 
0)  0 
0  0 
E  £ 
o  o 
5  Id 

3  3 

o  o 


LU 

CL 

3 

I- 

< 

CL 

LU 

CL 

2 

LU 


0 

x:  CM 
C  CO 
T3 
"D 
<3>  0 


in 


2 

0 

(0  Q. 

.2  E 
0 
0 
O 


PI 


ei 


Zi 


ii 


OL 


—  CO  CO 

£ - EE 


6  18  IZ  19 


u. 

p 

3 

o 

CM 

o 

-o 

CM_ 

- 

_o 

00 

o" 

co- 

-O 

(D 

o“ 

CM- 

q  ’ 

CO  ^ 
0)  - 

CO 

o_ 

“■ 

00 

o 

- 

CM 

Q. 

CM'^t 

-o 

o- 

o 

CM 

1 

_ 

u. 

“O 

o 

oj 

-  o 

1 

1 

M'l'l' 

‘I'I'l! 

'I'i'i' 

'I'I'I' 

'I'I'I' 

M'l'l' 

M'l'l' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

'I'I'I' 

9 

8 

7 

6 

5 

4 

3 

2 

1 

ir 

iches 

CO 

0 

3 

CO 

CO 

0 


o 

'k- 

0 


0 

c 

o 

0 

k» 

0 

> 

c 

o 

o 

0 

£ 

x 

o 

V. 

Q. 

a 

< 


o 

X3 

E 

> 

0 


X) 

c 

ll 


>k 

m 

a 


$ 

o 

c 

C 

0) 

xz 

5 


o 

X3 

E 

>s 

0 


£  E  p  E 
0  0*=^ 


t>^  CO 


c  C\J  CM 
0  £  £ 


E  0 


—  —  —  CO  CO 

SEE - EE 


w 


0  0 
0  0 
£  E 


«  0) 
0  E 
o 


c  C  fl.  w 

0  0)  y  = 

CJ  o  £  Jct 


in  q  CO 
cvi  o  d  T-^ 

I*  " 


0 

0  0 
c  0  0  -ii' 

C  k.  k-  m 
Q  0)  g 
m  0  0  O 

See;! 

0  0  0  <13 

0  0  0  CO 


3  O 
CJ  CJ  0 
W  to  jC 


a> 

qqqq'tj 
CD  d  d  CM  d 


0 

E  • 

0  W 

e  5^  0 
E  CT  c 
0  o  c 


in 

^  O) 

00  d  cj 

CM 


LU 

3 

o 

> 


0 
0 

,E  ^  ^  E 


o 

C  0 


^  T3  0 


—  T3 


0  0  -p 


ra  = 
>>  E 


.s  r 


0  0  0  0  0} 

3  3  3  Di- 

cr  CJ  CJ  CJ  0 

0  0  0  0  0 


o 

o 

o 

CM 


0  0  2 
8E| 

i  or 
o  a  o 

JZ 

0 


6  ^ 


0  0  0 
0  0  0 

j*;£  •=  i=  0  0  0  0 

1^  Ie  0  0  0  0 

'  £  £  E  S  {E  S 


0  2 
0  0 
0  0 
E  £ 
o  o 
15  Id 

3  3 

o  o 


^  in  CO  CO 

q  q  q  q  rN. 

inmo  dddwcJd 

^  CO 


o 

< 

X 

LU 

LU 

CL 

3 

H 

< 

CL 

lU 

0. 

{i!i 


0  0 
C  0 

2  ^ 
o  c 
a  3 
0  o 
0  — 

jQ  12  a  ^  "*  "3  — 

3  3  .£  3  0  3 

2  =  o  a  cr  C3)  0 


0  2 
«  0  “C  o  .2 
“  c  2  ^  -2 


O 


0 

0 

.3  E 

J0  0 
0 
o 


0.E 

|| 
0)2 
■>.  3  CM 
m  0  CO 


0 

3 

0 


0 


0 
a 
c  E 
0  0 


x: 

0 

U. 


a  N 
a  0  o 


a  w  tj  ^ 

0  jQ  0  ^  ^ 

^  a  zz  o  aao)-*-  > 


IV 


ACKNOWLEDGEMENTS 


Many  people  provided  encouragement,  guidance,  and  suggestions  during  this  complex 
study.  The  authors  learned  from  them  all  and  are  grateful  to  them  all,  including: 

•  the  planners  and  guides.  Dr.  Marc  B.  Mandler  and  Dr.  Lee  Alexander 

•  the  U.S.  Coast  Guard  sponsor,  CAPT  Leo  Black 

•  the  Maritime  Administration’s  representative,  Mr.  Frederick  Siebold 

•  the  human  factors  consultants.  Dr.  John  O'Hara  and  Dr.  William  S.  Brown 

•  the  master  mariners,  Capt.  Frank  Seitz,  Capt.  Joesph  P .  Maco,  and 
Capt.  William  R.  Daniels 

•  the  hydrographers,  Mr.  R.  Mike  Eaton  and  Mr.  Frederick  K.  Ganjon 

•  the  U.S.  Coast  Guard  reviewers,  CDR  David  M.  Loerzel  and 
Mr.  Albert  W.  Hartberger 

The  authors  would  also  like  to  thank  the  manufacturers  of  the  ECDIS  devices  for  their 
cooperation  during  the  study: 

•  Offshore  Systems  Limited,  Vancouver,  Canada 

•  Robertson  Marine  Systems,  Incorporated,  Egersund,  Norway 


4 


v/vi 


EXECUTIVE  SUMMARY 


INTRODUCTION 

In  the  last  few  years,  the  Electronic  Chart  Display  and  Information  System  (ECDIS)  has 
emerged  as  a  powerful  addition  to  the  modem  bridge,  offering  the  possibility  of  effecting 
major  changes  in  the  navigation  process  and  improving  the  safety  and  efficiency  of 
maritime  operations.  By  superimposing  chart,  ship's  real-time  position,  and  radar  on  one 
display,  ECDIS  has  the  potential  to  improve  the  accuracy  of  navigation,  increase  awareness 
of  dangerous  conditions,  and  reduce  the  mariner’ s  workload.  This  report  describes  an 
examination  of  these  potential  effects  using  the  special  capabilities  of  a  full-mission  ship’s 
bridge  simulator. 

PURPOSE 

There  were  two  purposes  for  conducting  this  study.  One  was  to  contribute  to  the  United 
States’  position  on  the  International  Maritime  Organization’s  (IMO)  Performance  Standards 
(PS)  for  ECDIS.  Another  was  to  examine  the  contributions  that  the  new  ECDIS 
technology  might  make  to  operational  practices  on  the  commercial  ship’s  bridge,  to  identify 
the  conditions  under  which  these  contributions  will  be  made,  and  to  identify  developments 
needed  before  ECDIS  can  make  its  best  potential  contribution. 

OBJECTIVES  AND  SCOPE 

The  study  was  designed  to  consider  the  following  four  issues  during  the  dynamic  process 
of  route  monitoring: 

•  The  potential  of  ECDIS  to  contribute  to  safe  navigation 

•  The  potential  of  ECDIS  to  reduce  the  navigational  workload 

•  The  chart  features  and  navigational  functions  required  by  the  mariner 

•  The  potential  contribution  of  the  integration  of  radar  features  on  ECDIS 

TECHNICAL  APPROACH 

Two  commercially-available  ECDIS  devices  were  installed  on  the  simulator  bridge  at 
MarineSafety  Intemational/Computer  Aided  (derations  Research  Facility  (MSI/CAORF)  in 
Kings  Point,  New  York.  The  two  devices  offered  different  user  interfaces,  chart 
presentations,  and  radar  integration  for  the  mariners’  inspection  and  use. 


Expert  mariners  made  repeated  port  arrivals  and  departures,  using  either  conventional 
navigation  procedures  or  one  of  the  two  EQDIS  devices.  The  EDCIS  devices  were 
configured  to  operate  in  one  of  three  modes  -  with  automatic  position  updating  and  radar 
features,  with  automatic  updating  and  no  radar  features,  or  without  automatic  position 
updating  and  with  the  instmctions  to  update  manually.  A  variety  of  performance  measures 
were  collected,  including  ship  position  data,  observer  counts  of  ECDIS  features  used  and 
navigational  errors  made,  mariner  ratings  of  perceived  workload,  and  mariner  reports  of 
transit  events.  Extensive  debriefings  were  conducted  in  order  to  collect  reactions  and 
recommendations  from  these  expert  mariners,  based  on  their  just-completed  experience 
with  ECDIS. 

PRINCIPAL  FINDINGS 
Contribution  to  Safe  Navigation 

ECDIS  has  the  potential  to  improve  upon  the  safety  of  navigation,  compared  to  con¬ 
ventional  procedures.  There  was  strong  evidence  that  the  use  of  ECDIS  increased  the 
accuracy  of  navigation,  as  measured  by  a  smaller  cross-track  distance  of  the  ship  from  the 
planned  track  line,  and  reduced  the  proportion  of  time  spent  on  navigation,  with  a 
corresponding  increase  in  the  proportion  of  time  spent  on  the  higher  risk  collision 
avoidance  task.  In  addition,  ECDIS  was  shown  to  improve  geographic  “situational 
awareness,  ”  and  to  reduce  navigation  “errors.” 

Reduction  in  Mariner  Workload 

The  strongest  and  most  consistent  finding  was  that  the  availability  of  ECDIS  on  the  bridge 
substantially  reduces  the  mariner’s  workload  for  navigation.  Mariner  ratings  of  workload 
decreased  significantly  with  ECDIS,  they  used  a  smaller  proportion  of  time  on  navigation, 
and  made  spontaneous  comments  such  as:  “Navigation  goes  away  as  a  task.” 

The  ECDIS  devices  had  no  effect  on  the  workload  for  collision  avoidance  or  for  bridge 
management.  In  order  to  reduce  workload  for  these  tasks,  ECDIS  must  have  a  user 
fiiendly  integration  of  all  Automatic  Radar  Plotting  Aid  (ARPA)  features  and  must  automate 
such  functions  as  gear  testing  and  record-keeping. 
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Mariners’  Requirements  for  System  Characteristics,  Chart  Features  and 
Navigational  Functions 

This  study  identified  a  number  of  system  characteristics  required  for  the  effective  use  of 
ECDIS  for  route  monitoring  in  the  Coastal  and  Harbor/Harbor  Approach  phase  of 
navigation:  all  charted  information  must  be  accurate,  updating  of  own  ship’s  position  must 
be  accurate  and  automatic,  scaling  of  own  ship  in  narrow  channels  must  be  accurate,  and  a 
selected  subset  of  chart  features  must  be  always  available.  The  safety  of  ship  control 
requires  a  detailed  and  accurate  view  of  the  immediate  surround.  In  addition,  a  larger  set  of 
chart  features  must  be  available  for  reference  and  navigation  functions  must  be  implemented 
in  a  user-friendly  manner. 

Two  sets  of  requirements  for  chart  features  were  identified.  For  the  dynanuc  function  of 
route  monitoring,  only  a  very  simple  subset  of  chart  features,  features  that  outline  the  safe 
water  available  for  the  transit  was  required.  These  included:  coastline/  landmass,  fixed  and 
floating  aids  to  navigation,  federal  channel  lines,  navigation  lanes  and  fairways,  and 
isolated  dangers.  These  features  arc  all  in  the  “display  base”  or  “standard  display”  of  the 
IMO  Performance  Standards.  A  much  larger  subset  of  charted  features  was  seen  as  serving 
a  static  function  as  a  geographic  information  system  (GIS),  available  for  reference  as 
needed  but  not  cluttering  the  screen.  Examples  of  these  were:  soundings,  depths,  bottom 
contours,  aids  to  navigation  characteristics,  names  of  points,  etc. 

Mariners  recommended  that  navigation  functions,  such  as  planned  track  line  or  own  ship 
outline,  be  developed  with  the  inclusion  of  the  end  user,  that  they  not  lead  to  a  system  that 
is  overly  complicated,  cluttered  and  confused;  and  that  there  be  sufficient  standardization  so 
that  an  experienced  mariner  can  make  immediate,  effective  use  of  a  different  system. 

Contribution  of  Radar  Integration 

The  smdy  also  examined  the  potential  contribution  that  the  integration  of  ECDIS  and  radar 
might  make  to  navigation  performance,  beyond  that  made  by  ECDIS  used  with  a  separate 
radar.  This  study  found  that  the  integration  of  radar  features  on  the  experimental  ECDIS 
devices  had  no  measurable  effect  on  the  safety  of  navigation.  Mariners  expressed  an 
increase  in  confidence  in  the  accuracy  of  positioning  but  no  effects  on  trackkeeping  or  on 
any  measure  of  navigation  workload  were  found. 

The  traffic-dense  port  arrivals  and  departures  required  frequent  attention  to  cpHisiQh 
avoidance.  One  of  the  ECDIS  devices  provided  a  full  radar  overlay  and  the  other  provided 
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only  ARPA  targets.  Both  provided  some  ARPA-type  information.  Mariners,  however, 
made  heavy  use  of  the  separate  radar/ARPA  on  the  bridge  in  all  conditions,  reporting  that 
only  the  complete  ARPA  was  adequate  for  their  needs.  The  integrated  systems  were  not 
adequate  and  no  reduction  in  the  workload  of  collision  avoidance  was  found. 

To  achieve  a  more  effective  integration  of  ECDIS  and  radar,  it  is  recommended  that  users 
have  the  option  of  selecting  between  display  of  complete  radar  video  and  ARPA  targets 
only,  that  complete  ARPA  information  be  available,  that  the  screen  be  kept  uncluttered  by 
simplifying  the  chart  display  and  by  minimizing  sea/rain  clutter. 

ECDIS  as  Automation 

In  the  words  of  one  of  the  participating  mariners,  “The  units  are  more  than  replacements  for 
the  chart  —  they  are  replacements  for  human  activity.”  Many  of  the  ECDIS  issues 
considered  during  this  study  are  generic  to  the  use  of  automated  systems.  These  include 
the  effects  of  automation  on  safety,  on  workload,  on  situational  awareness,  etc.  Because 
of  the  broad  implications  of  such  issues  for  maritime  safety,  the  USCG  is  involved 
presently  in  a  major  study  of  the  effects  of  shipboard  automation  on  human  performance. 

A  broad  range  of  activities  to  examine  changes  in  mariner  functions,  including  user- 
interface  evaluation,  workload  studies,  requirements  analyses,  and  training  and 
qualifications  analyses,  will  provide  the  USCG  with  the  data  and  procedures  that  will 
support  all  stages  of  the  system  design  and  regulatory  development  process. 
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1.0  INTRODUCTION 

1 . 1  EXAMINATION  OF  THE  MARINER’S  USE  OF  THE  ELECTRONIC 
CHART  DISPLAY  AND  INFORMATION  SYSTEM  (ECDIS) 

1.1.1  Examination  of  a  New  Technology 

In  the  last  few  years,  the  Electronic  Chart  Display  and  Information  System  (ECDIS)  has 
emerged  as  a  powerful  addition  to  the  modem  bridge,  offering  the  possibility  of  effecting 
major  changes  in  the  navigation  process  and  improving  the  safety  and  efficiency  of  mar¬ 
itime  operations.  By  superimposing  chart,  ship's  real-time  position,  and  radar  on  one 
display,  ECDIS  has  the  potential  to  improve  the  accuracy  of  navigation,  increase  awareness 
of  dangerous  conditions,  and  reduce  the  mariner’s  workload.  Because  this  is  a  new 
technology,  there  are  complex  engineering,  operational,  and  human  operator  issues  to  be 
explored  before  ECDIS  can  make  its  maximum  contribution.  This  report  describes  an 
examination  of  the  human  operator’s  use  of  ECDIS,  employing  the  special  capabilities  of  a 
full-mission  ship’s  bridge  simulator. 

1.1.2  Purpose 

The  United  States  Coast  Guard’s  (USCG)  first  purpose  in  conducting  this  study  was  to 
contribute  to  the  United  States’  position  on  the  International  Maritime  Organization’s  (IMO) 
Performance  Standards  (PS)  for  ECDIS  (International  Maritime  Organization,  1989, 

1994). 

A  second  purpose  is  to  collect  operator-in-the-loop  performance  data  on  a  shiphandling 
simulator  to  determine  how  an  ECDIS  will  be  used  on  a  bridge,  to  determine  the  situations 
in  which  it  enhances  navigation,  and  to  determine  further  design  developments  needed  for 
wide-scale  use  in  the  maritime  industry. 

A  secondary  purpose  was  a  methodological  one:  to  select  and  examine  the  procedures  and 
performance  measures  that  might  be  used  in  later,  related  simulator  studies  or  in  sea  trials. 


1.1.3  Objectives  and  Scope 


The  IMO  Performance  Standards  were  reviewed  and  several  issues  were  selected  for 
examination.  Criteria  for  the  selection  were  that  the  issues  be  very  central  to  the  Standards, 
that  they  have  broad  implications  for  operational  use  of  ECDIS,  and  that  the  shiphandling 
simulator  be  an  appropriate  tool  for  their  investigation.  The  study  was  designed  to  consider 
the  following  four  issues  during  route  monitoring. 

1 .  The  potential  of  ECDIS  to  contribute  to  safe  navigation  (IMP  PS.  Item  1.1).  To 
allow  the  demonstration  of  potential  effects  on  safety,  scenarios  were  designed  with 
the  relatively  high  risk  conditions  of  port  arrivals  and  departures  with  surrounding 
traffic  and  narrow  channels.  ECDIS  should  provide  equal  or  better  quality  and 
timeliness  of  information  to  the  mariner  for  the  control  of  the  vessel,  compared  to 
conventional  methods  of  navigation.  The  quality  of  information  provided  was 
evaluated  by  comparing  the  precision  of  navigation  (trackkeeping)  using  ECDIS,  with 
performance  using  conventional  methods. 

2.  The  potential  of  ECDIS  to  reduce  the  navigational  workload  (IMO  PS.  Item  1.61. 

The  potential  of  ECDIS  to  reduce  workload  was  examined  under  the  relatively 
demanding  conditions  of  a  one-officer  bridge  during  arrivals  and  departures.  A 
primary  value  of  an  automated  system  should  be  the  reduction  of  operator  workload, 
compared  to  conventional  methods  of  navigation.  A  reduction  in  workload  suggests 
an  increase  in  safety  as  the  mariner  has  more  time  for  monitoring  and  “look-out.” 

To  evaluate  the  importance  of  an  automatic  positioning  input  to  a  potential  reduction 
in  workload,  scenarios  with  a  “failure”  of  the  positioning  system  were  included. 

3.  The  chart  features  and  navigational  functions  required  by  the  mariner  flMO  PS. 

Items  1.4  and  3  and  Appendix  21.  Whether  ECDIS  will  contribute  to  safe 
navigation  and  reduce  workload  depends  on  whether  all  the  required  chart  features 
and  navigational  functions  are  available  to  the  mariner.  The  availability  of  the 
required  features  and  functions  is  especially  critical  during  the  real-time  operation  of 
route  monitoring.  The  mariners’  use  of  features  and  functions  available  on  the 
experimental  systems  was  observed  during  the  scenarios  and  the  mariners  were 
questioned  about  a  more  extensive  list  of  potential  items. 
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4.  The  potential  rontrihiirion  of  the  integration  of  radar  features  on  ECDIS  (IMP  ES. 
Ttem  6~>.  The  integration  of  electronic  chart,  vessel  position,  and  radar  on  one  scope 
offers  the  possibility  of  greater  awareness  of  dangerous  conditions  and  further 
reduction  of  mariner  workload.  There  is  as  yet  no  agreement  as  to  what  such  a 
system  should  look  like.  The  experiment  evaluated  performance  with  both  an  overlay 
of  the  complete  radar  video  and  a  radar  overlay  that  contained  only  ARPA  targets. 

Planning  during  Phase  I  of  the  reported  project  involved  a  literature  search  and  the 
generation  of  an  exhaustive  catalogue  of  questions  concerning  the  utility  and  implications  of 
a  wide  variety  of  ECDIS  features  and  functions.  This  process  is  described  in  "Proposal  for 
an  ECDIS  Research  Program,"  (MSI/CAORF  staff,  1991) 

1.2  BACKGROUND 

1.2.1  Tntpmational  Maritime  Organization’s  (IMP)  Performance  Standards  fpr  ECDIS. 

In  1989  the  IMO  issued  Provisional  Performance  Standards  (PPSl  for  Electronic  Chart 
Display  and  Information  Systems  (ECDIS)  (International  Maritime  Organization,  1989), 
and  allowed  time  for  participating  nations  to  eyaluate  these  standards  and  to  submit 
recommendations  on  their  adequacy.  The  USCG  goal  in  sponsoring  this  simulator 
evaluation  was  to  contribute  to  the  U.S.  position  on  the  Standards.  The  1989  version  was 
a  major  consideration  in  the  planning  of  the  present  study.  In  1994,  the  Performance 
Standards  fPSl  for  Electronic  Chart  Display  and  Information  Systems  (ECDISl 
(International  Maritime  Organization,  1994)  was  issued.  Because  the  study  was  planned 
around  larger  issues  addressed  by  the  Standards,  the  findings  are  as  relevant  to  the  1994 
version  as  they  were  to  the  1989  version.  A  summary  of  the  experiment’s  findings  on  the 
Standards  is  presented  in  Appendix  F  of  the  present  report. 

1.2.2  Tntp.marional  Maritime  Organization's  flMOl  Annex  1  List  of  Matters  for  Usg  in_ 
Examining  Subject  Areas  Which  Mav  Have  Human  Factor  Implications 

The  IMP’S  Marine  Safety  Committee  has  distributed  Annex  1  List  of  Matters  for  Use  in 
Examining  Subject  Areas  Which  May  Have  Human  Factor  Implications  (International 
Maritime  Organization,  April  1992)  to  its  various  Sub-Committees.  The  IMO  Human 
Factors  List  is  intended  as  a  diagnostic  tool,  to  be  used  to  examine  a  system  or  procedure  to 
determine  whether  it  has  human  factors  implications  that  might  require  further  study.  The 
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factors  included  in  the  List  for  consideration  were  reviewed  for  application  to  the  mariner’s 
use  of  ECDIS  for  watchstanding  (International  Maritime  Organization,  August  1992). 

1.2.3  United  States  Coast  Guard's  Integrated  Navigational  Systems  (INS)  Program 

The  human  factors  study  reported  here  is  a  component  of  the  comprehensive  Research, 
Development,  Test  and  Evaluation  (RDT&E)  program  being  conducted  by  the  USCG 
Research  and  Development  (R&D)  Center  to  assess  the  operational  capabilities  and  limita¬ 
tions  of  ECDIS.  The  Office  of  Coast  and  Geodetic  Survey  (C&GS)  at  the  National  Ocean 
Service,  NOAA  and  USCG  are  leading  participants  in  a  joint  government-industry  research 
project  to  evaluate  the  Standards  for  ECDIS  (International  Maritime  Organization,  1994). 

In  addition,  the  USCG  is  evaluating  the  Radio  Technical  Commission  for  Maritime 
Services  (RTCM)  Recommended  Standard  for  Electronic  Chart  Systems  (ECS)  (Radio 
Technical  Commission,  1994). 

Various  prototypes  and  commercially  available  systems  have  been  tested.  During  October 
1991,  the  USCG  R&D  Center  conducted  a  series  of  at-sea  experiments  onboard  the  U.S. 
Merchant  Marine  Academy  TA'^  KINGS  POINTER  (Gonin  and  Crowell,  1992).  Between 
January  and  April  of  1993,  the  USCG  R&D  Center  conducted  human  factors  sea  trial  tests 
using  ECDIS  aboard  the  USCG  Cutter  BITTERSWEET  and  the  new  Motor  Vessel 
KINGS  POINTER  (Gonin  and  Dowd,  1994).  Additional  at-sea  evaluations  are  planned 
onboard  a  variety  of  vessels  and  various  USCG  cutters/boats.  A  joint  research  program 
agreement  (JRPA)  has  been  established  between  the  United  States  and  Canada  to  conduct 
joint  RDT&E  of  ECDIS  and  ECDIS-related  technologies  (Alexander  and  Casey,  1992). 
Under  this  U.S. /Canada  JRPA,  a  small  scale  human  factors  simulator  test  using  ECDIS 
was  conducted  at  the  Centre  for  Marine  Simulation  (CMS)  in  St.  John's  Newfoundland 
(Center  for  Marine  Simulation,  1994). 

The  findings  of  the  U.S.  RDT&E  Program  on  ECDIS  supported  the  development  of  the 
IMG  Standards  for  ECDIS  and  will  continue  to  support  the  development  and  operational 
implementation  of  this  important,  new  technology. 
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1.2.4  ParliftT  Tnvestipations  of  the  Use  of  Electronic  Navigation  Systems 


The  USCG  has  had  an  active  human  factors  research  program  under  way  for  more  than  a 
decade  to  examine  the  potential  effects  of  new,  developing  technologies  on  navigational 
performance.  The  program  has  focused  on  the  effectiveness  of  electronic  displays  for  the 
specific  operation  of  harbor/harbor  approach  piloting,  especially  in  reduced  visibiUty. 
Systems  have  been  examined  both  at  sea  (Cooper  and  Bertsche,  1981;  Roeber,  1981)  and 
on  real-time  man-in-the-loop  shiphandling  simulators.  These  investigation  provided  many 
of  the  experimental  procedures  and  measures  discussed  in  Section  4  of  this  report 

A  review  of  the  earlier  studies  also  provided  hypotheses  on  the  performance  effects  to  be 
expected  with  ECDIS .  It  has  been  demonstrated  that  the  precision  of  navigation  and 
mariner  acceptance  of  a  new  technology  were  very  sensitive  to  differences  in  specific 
display  features  (Smith,  1993;  Mandler  and  Smith,  1990;  Mandler,  Smith,  and  Gynther, 
1990-1991;  Smith,  Mandler,  Mazurkiewicz,  Gynther,  and  Brown,  1990;  Gynther  and 
Smith,  1989;  Cooper,  Marino  and  Bertsche,  1981a;  Cooper  and  Marino,  1980).  The 
expected  importance  of  specific  features  was  one  reason  for  including  two  different 
commercial  systems  in  the  experiment.  Together,  they  gave  the  marines  the  opportunity  to 
examine  and  use  a  wider  range  of  features  than  either  alone.  Earlier  studies  demonstrated 
the  influence  of  positioning  accuracy  on  the  effectiveness  of  navigation  systems  in 
supporting  ship  control  and  on  mariner  reaction  (Gynther  and  Smith,  1989,  Cooper, 
Marino,  and  Bertsche,  1981b).  While  the  accuracy  of  the  positioning  input  was  not  varied 
in  this  experiment,  these  earlier  findings,  together  with  the  negative  reactions  of  mariners  in 
this  experiment  to  such  inaccuracies  as  digitizing  errors  in  the  electronic  chart  and  unsealed 
ship  icons,  support  a  conclusion  that  display  accuracy  is  important  to  ECDIS  function. 
Findings  of  earlier  studies  provided  an  expectation  that  mariners  would  value  a  radar 
overlay,  to  monitor  both  positioning  accuracy  and  the  action  of  traffic  (Smith,  1993).  The 
relevance  of  many  of  these  past  investigations  of  electronic  navigation  to  ECDIS  design 
was  reviewed  in  a  recent  paper  (Smith  and  Mandler,  1992). 

The  Computer  Aided  Operations  Research  Facility,  now  MarineSafety  International/ 
Computer  Aided  Operations  Research  FacUity  (MSI/CAORF),  has  a  history  of  experimen¬ 
tation  in  the  mariner’s  use  of  advanced  bridge  equipment.  Examples  include  investigations 
of  the  use  of  displays  for  ship  positioning  and  navigation  (CAORF  Research  Staff,  1978a) 
and  for  collision  avoidance  (Schryver,  1983;  Aranow,  1979;  Hayes,  1979;  CAORF 
Research  Staff,  1978b).  More  complex  integrated  displays  have  also  been  investigated 
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(Williams,  Goldberg,  and  Nieri,  1982;  Hayes  and  Wald,  1980).  In  some  cases,  these  de¬ 
veloping  technologies  were  applied  to  specific  operational  problems  (O’Hara  and  Brown, 
1985).  These  CAORF  studies  also  contributed  to  the  hypotheses  and  the  experimental 
procedures  of  the  present  study. 

1 . 3  USE  OF  A  SHIPHANDLING  SIMULATOR  FOR  SYSTEM  EVALUATION 

1-3.1  Simulator  Evaluation  of  System  Performance 

A  full-mission  shiphandling  simulator  is  a  powerful  tool  for  the  evaluation  of  ECDIS.  The 
use  of  simulation  to  evaluate  operator-machine  system  performance  in  the  marine  environ¬ 
ment  is  a  well-established  practice  (Smith,  1993;  Smith  and  Mandler,  1992;  O'Hara  and 
Brown,  1985;  Reik  and  Hargis,  1981).  Simulation,  when  used  as  a  test  platform,  provides 
many  capabilities  not  possible  in  real-world  evaluations.  These  include  the  capability:  to 
concentrate  only  on  events  of  high  interest,  to  control  all  aspects  of  the  external 
environment,  to  replicate  tests,  to  examine  relatively  high-risk  conditions,  and  to  monitor 
and  record  many  aspects  of  system  and  human  performance.  The  ECDIS  test  plan  was 
intended  to  maximize  the  advantages  of  a  simulator  test  bed  by  selecting  those  issues  most 
effectively  or  efficiently  examined  there.  The  simulation  exercises,  therefore,  consisted 
primarily  of  route  monitoring  in  the  relatively  high  risk  coastal  and  harbor/harbor 
approach  phases  of  navigation  during  vessel  arrivals  and  departures  from  port  (Federal 
Radionavigation  Plan,  1992).  Passage  planning  was  addressed  only  by  having  the  partici¬ 
pating  mariners  perform  a  brief  exercise  and  consider  such  a  use  based  on  their  experience 
with  the  systems  during  the  route  monitoring  simulation  tests.  Engineering  aspects  of 
ECDIS  technology  and  those  human  factors  issues  better  evaluated  through  longer  mariner 
exposure  to  the  technology  were  not  considered.  Thus,  topics  like  system  reliability,  chart 
correction,  and  vigilance  were  excluded  from  the  simulation  evaluation  on  the  assumption 
that  they  could  better  be  evaluated  during  real-world  test  bed  trials. 

1.3.2  Use  of  the  MarineSafety  Intemational/Computer  Aided  Operations  Research 
Facility.  MSI/CAORF 

The  MSI/CAORF  simulator  has  many  capabilities  for  the  effective  conduct  of  human 
factors  evaluations  of  vessel  operations.  As  an  example,  the  simulator  facility  is  equipped 
with  a  human  factors  monitoring  station  from  which  an  experimenter  can  closely  watch  and 
record  activities  on  the  bridge  and  from  which  all  simulation  parameters  can  be  recorded  for 
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later  detailed  analysis.  In  addition,  the  modular  design  of  the  bridge  permits  re¬ 
configuration  of  the  bridge  instrumentation.  The  resident  engineering  staff  was  available 
for  the  required  integration  of  the  commercial  ECDIS  devices  with  the  existing  bridge  suite 
during  the  first  phase  of  this  project  (MSI/CAORF  staff,  1991).  This  integration  required 
staff  engineers  and  data  base  programmers  who  were  familiar  with  the  simulation  systems, 
the  ECDIS  technology,  and  the  project  objectives.  A  more  detailed  description  of  CAORF 
capabilities  appears  in  Appendix  A. 

1 .4  OVERVIEW  OF  THE  REPORT 

The  remainder  of  this  report  is  organized  as  follows: 

SECTION  2.0  SIMULATION  AS  A  "TEST  BED"  FOR  ECDIS  EVALUATION  de¬ 
scribes  briefly  the  two  commercial  ECDIS  devices  that  were  selected  and  the  configuration 
of  the  MSI/CAORF  simulator  for  the  experiment.  Both  the  systems  and  the  simulator  fea¬ 
tures  used  in  the  experiment  are  described  further  in  Appendix  B. 

SECTION  3.0  EVALUATION  PLAN  includes  a  description  of  the  experimental  design, 
the  scenarios,  and  the  participating  mariners.  Samples  of  the  instructions  are  presented  in 
Appendix  C. 

SECTION  4.0  PERFORMANCE  MEASUREMENT  describes  the  variety  of  performance 
measures  that  were  selected  or  developed  to  benefit  from  the  capabilities  of  the  simulator  in 
examining  the  complex  task  of  watchstanding.  Additional  discussion  and  sample  data 
collection  forms  and  questionnaires  are  presented  in  Appendix  D. 

SECTION  5.0  RESULTS  AND  DISCUSSION:  CONTRIBUTION  OF  ECDIS  TO 
SAFETY  AND  WORKLOAD  IN  ROUTE  MONITORING  is  the  primary  result  section.  It 
presents  and  discusses  the  findings  comparing  route  monitoring  using  ECDIS  to  route 
monitoring  done  using  conventional  methods.  Both  simulator  measures  and  mariner 
reactions  are  reported.  Additional  data  not  included  in  the  discussion  in  Section  5.0  are 
presented  in  Appendix  E. 

SECnON  6.0  RESULTS  AND  DISCUSSION:  ROLE  OF  ECDIS  FEATURES  IN 
ROUTE  MONITORING  presents  findings  on  the  use  of  chart  features  and  ECDIS-based 
navigation  functions.  Both  experimenter  observations  and  mariner  creactions  are  reported. 
Additional  material  is  presented  in  Appendix  E. 
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SECTION  7.0  RESULTS  AND  DISCUSSION:  ROLE  OF  ECDIS-BASED  RADAR 
FEATURES  IN  ROUTE  MONITORING  presents  findings  on  the  use  of  ECDIS  radar 
features  during  route  monitoring.  Additional  material  is  presented  in  Appendix  E. 

SECTION  8.0  CONCLUSIONS  contains  conclusions  that  follow  from  the  results  in 
Sections  5.0, 6.0,  and  7.0.  A  table  containing  an  item-by-item  listing  of  the  IMO  PS  ac¬ 
companied  by  a  summary  of  relevant  findings  for  each  item  appears  in  Appendix  F. 
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2.0 


SIMULATION  AS  A  "TEST  BED"  FOR  ECDIS  EVALUATION 


2. 1  SELECTION  OF  REPRESENTATIVE  ELECTRONIC  CHART  DISPLAY  AND 
INFORMATION  SYSTEMS  (ECDIS) 

Phase  I  of  this  project  (MSVCAORF  Staff,  1991)  included  a  survey  and  evaluation  of 
existing,  conunercially  available  ECDIS  systems  to  identify  those  most  appropriate  for  use 
in  this  human  factors  evaluation.  It  should  be  emphasized  tiiat  the  objective  of 
incorporating  existing  systems  into  the  simulator  experiment  was  not  to  evaluate  any 
particular  systems  but  to  allow  the  mariners  to  examine  and  to  use  a  number  of  possible 
configurations  and  features  during  real-time  bridge  operations.  The  criteria  used  in 
assessing  the  appropriateness  of  each  candidate  system  were:  ECDIS  features,  company 
profile,  simulator/ECDIS  integration,  cost,  and  availability  and  delivery  schedule.  Two 
systems  were  selected: 

.  Offshore  Systems  Ltd.’s  fOSLl  Precision  Inteerated  Navigation  Svsigm  -  YME 
fPTNS-VMEl  This  system  provided  a  demonstrated  operational  radar  overlay 
capability.  It  was  procured  by  the  Coast  Guard  for  both  this  human  factors 
evaluation  and  later  sea  trials. 

•  Robertson  Marine  Systems  Inc.’s  Disc  Navigation  System.  This  system  most  closely 
conformed  to  the  IMO  Provisional  Performance  Standards  (International  Maritime 
Organization,  1989)  at  the  time  of  the  selection.  Robertson  Marine  Systems,  Inc. 
loaned  their  Disc  Navigation  System  to  MSI/CAORF  for  the  duration  of  the  human 
factors  study. 

2.2  DESCRIPTION  OF  THE  SELECTED  ECDIS  DEVICES 

The  Offshore  Systems  Limited’s  (OSL)  Precision  Integrated  Navigation  System  (PINS- 
VME)  and  Robertson  Marine  Systems  Incorporated’s  Disc  Navigation  System  differed 
from  each  other  in  many  ways,  the  most  major  of  which  are  summarized  in  Table  2-1.  The 
OSL  system  had  a  single  screen  that  could  be  configured  by  the  user  to  present  several 
graphic  and  alphanumeric  windows;  the  Robertson  system  had  a  screen  dedicated  to  the 
chart  presentation  and  a  separate  liquid  crystal  display  (LCD)  to  present  alphanumeric  in¬ 
formation.  The  OSL  had  relatively  simple,  stylized  charts  that  could  be  viewed  in  separate 
windows  at  different  scales;  the  Robertson  had  a  more  complex,  “chart-like”  chart  pre 
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Table  2- 1 .  Major  Differences  Between  the  Two  Commercial  ECDIS  Devices 


COMMERCIAL 

Systems 

Offshore  Systems 
Limited 
PINS-VME 

Robertson  Marine 
Disc  Navigation 

Computer 

68030  @25  MHz 

80386  @33  MHz 

Screen 

19  inch,  1024  x  788  pixels 

25  inch,  1080  x  1040  pixels 

DISPLAYS 

configurable  into  windows, 
chart  or  text 

chart  on  screen, 
text  on  LCD 

Electronic 

Chart 

landmass,  contours, 
channels,  aids 

complex,  “chart-like” 

Interface 

touch  screen  and  trackball 

keyboard  and  trackball 

RADAR  Features 

video  overlay, 
some  ARPA  info 

ARPA  targets, 
some  ARPA  info 

Own  Ship  Symbol 

scaled  outline 

outline  not  to  scale 

sented  on  the  single  screen.  The  OSL  presented  complete  radar  video  as  an  overlay  to  the 
chart  and  presented  target  range  and  bearing  information  in  an  alphanumeric  window;  the 
Robertson  system  presented  only  the  targets  acquired  by  the  separate  ARPA  on  the  bridge 
and  their  vectors  on  the  chart  display  and  presented  range,  bearing,  closest  point  of 
approach  (CPA),  and  time  to  CPA  on  the  LCD. 

These  systems  are  described  in  some  detail  in  Appendix  B.  Specific  features  are  described 
throughout  the  report  where  they  are  particularly  relevant  to  the  discussion.  The  systems 
are  described  as  they  were  configured  when  they  were  procured  and  throughout  the  Simula 
tor  tests.  Significant  upgrades  for  both  systems  became  available  during  the  course  of  the 
simulator  experiment.  However,  the  devices  were  not  upgraded  once  testing  began  in  or¬ 
der  to  maintain  the  integrity  of  the  experimental  design  and  to  ensure  continuity  between 
test  mariners.  Therefore,  the  system  capabilities  described  in  this  report  are  not  an  indica- 
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tion  of  either  devices'  present  capabilities.  This  is  particularly  true  for  the  OSL  system 
which  was  an  early  prototype  for  the  product  line. 

2.3  SIMULATOR  AND  SIMULATION  MODEL  CONFIGURATION 

The  MSVCAORF  simulator  has  a  variety  of  capabilities  that  make  it  appropriate  as  a  re¬ 
search  simulator.  It  has  a  realistically  equipped  bridge  from  which  the  mariner  looks  out 
onto  a  240-degree  computer  generated  visual  scene  of  the  area  being  navigated.  Bridge  in¬ 
strumentation,  including  radar/Automatic  Radar  Plotting  Aid  (ARPA),  Doppler  speed  log, 
revolutions  per  minute  (RPM)  indicator,  compass,  depth  sounder,  ship  s  wheel,  rudder 
angle  indicator,  etc.,  all  operate  in  coordination  with  the  visual  scene  to  recreate  a  highly 
realistic  and  familiar  environment  for  the  mariner.  In  the  background,  the  simulation  com¬ 
puters  accurately  recreate  the  vessel's  dynamics  and  its  interactions  with  the  simulated  envi¬ 
ronment  A  Human  Factors  Station  (HFS)  allows  all  bridge  activities  to  be  observed  and 
recorded.  All  aspects  of  the  simulation  itself  are  recorded  on  magnetic  tape  for  later  analy- 
sis. 

The  simulator  subsystems  that  played  an  important  role  in  the  formulation  and  conduct  of 
the  ECDIS  evaluation  included  the  simulation  geographic  model,  the  ship  response  model, 
the  simulator  bridge,  and  the  HFS.  The  role  and  configuration  of  each  of  these  subsystems 
are  described  in  Appendix  B. 


2.4  INFORMATION  TRANSFERRED  FROM  THE  SIMULATOR  TO  THE  ECDIS 

The  simulator's  main  computer  provided  data  which  both  ECDIS  devices  processed  as  if 
from  real-world  instrumentation.  This  included  data  that  mimicked  feeds  from  Differential 
Global  Positioning  System  (DGPS),  gyro  compass,  speed  log,  fathometer,  and  apparent 
wind  indicator  (the  latter  for  the  Robertson  only).  Each  device  received  radar  information 
that  mimicked  what  it  was  designed  to  receive  on-board  ship.  The  OSL  ECDIS  received 
raw  radar  data  from  the  simulator's  radar  signal  generator  which  it  used  to  generate  the 
radar  overlay  display.  The  Robertson  ECDIS  was  interfaced  with  the  Sperry  RASCAR 
Automated  Radar  Plotting  Aid  (ARPA )  and  received  data  on  targets  being  tracked  on  that 
device. 
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DGPS  failure  procedures  were  developed  for  scenarios  for  which  the  mariner  was  required 
to  manually  update  the  ECDIS  position.  Again,  different  procedures  were  needed  to 
produce  a  similar  “failure”  on  each  system.  For  the  Robertson  system  a  DGPS  failure  was 
simulated  by  disabling  DGPS  as  a  navigation  input  and  selecting  the  dead  reckoning  (DR) 
option.  The  OSL  did  not  have  a  DR  mode  so  the  simulator  was  programmed  to  calculate  a 
DR  position  using  the  ship's  heading  and  speed  through  the  water.  The  DR  positions  were 
sent  to  the  OSL,  which  processed  them  as  if  they  were  from  DGPS. 
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3.0  EVALUATION  PLAN 


3 . 1  ECDIS  RESEARCH  ISSUES  AND  THE  EVALUATION  PLAN 

In  recent  years,  as  the  ECDIS  technology  has  developed,  the  cognizant  national  and  inter¬ 
national  organizations  have  proposed  or  established  relevant  standards  (International 
Hydrographic  Organization,  1990;  International  Maritime  Organization,  1994  and  1989; 
Radio  Technical  Commission,  1989).  Early  versions  of  these  standards  and  a  variety  of 
industry  sources  were  examined  for  research  issues  during  Phase  I  of  the  Human  Factors 
Evaluation  of  ECDIS  (MSI/CAORF  Staff,  1991).  In  addition,  the  IMO’s  Human  Factors 
List  antemational  Maritime  Organization,  April  1992)  was  considered.  The  discussion 
here  emphasizes  the  IMO  Performance  Standards  (International  Maritime  Organization, 
1994  and  1989)  and  is  organized  by  those  principal  issues  described  in  this  report  in 
Section  1.1.3. 

1,  rnntrihiitinn  of  ECDIS  to  the  Safety  of  Navigation.  The  assumption  in  the  design  of 
this  experiment  was  that  the  use  of  ECDIS  for  navigation  would  enhance  safety  by 
affording  the  watchstander  an  accurate  and  more  timely  knowledge  of  the  ship’s 
position,  and  its  relation  to  a  planned  track  and  to  potential  hazards,  than  is  possible 
with  conventional  bridge  procedures  and  a  paper  chart.  To  examine  the  effect  of 
ECDIS  on  safety,  experienced  mariners  conducted  transits  under  a  variety  of  harbor, 
traffic,  and  environmental  conditions,  using  either  ECDIS  or  conventional 
procedures.  The  controlled,  experimental  nature  of  these  exercises  allowed  the 
identification  of  conditions  for  which  ECDIS  would  enhance  safety  compared  to  the 
conventionally  equipped  bridge. 

2.  Rpdncrinn  of  Navigational  Workload  bv  ECDIS.  ECDIS  can  integrate  information 
from  a  number  of  sensors  and  can  automate  the  primary,  and  generally  time- 
consuming,  navigation  function  of  position  fixing.  These  capabihties  should  reduce 
the  mariner’s  workload.  It  follows  that  a  “failure”  of  the  automatic  positioning 
feature  and  the  necessity  to  manually  update  the  position  of  ownship  on  the  ECDIS 
display  might  compromise  a  savings  in  workload.  There  is  an  implicit  assumption 
here  that  reduced  workload  means  greater  safety.  Established  human  factors 
procedures  were  used  to  collect  ratings  of  perceived  workload  from  the  participating 
mariners  for  the  conditions  they  experienced. 
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3.  Chart  Features  and ECDIS-based  Navigational  Functions.  ThelMOPS 
(International  Maritime  Organization,  1994, 1989)  lists  required  chart  features  and 
ECDIS-based  navigation  functions.  The  simulator  experiment  allowed  the  evaluation 
of  these  requirements  by  observing  the  mariners’  actual  selection  of  features  and 
functions  from  the  two  sample  devices  under  a  variety  of  conditions.  Extensive 
debriefing  of  the  participating  mariners  documented  their  preferences  on  ECDIS 
features  and  allowed  the  extrapolation  to  features  that  were  not  provided  by  the 
experimental  devices. 

4.  Integration  of  Radar  Features  on  ECDIS.  The  issue  of  whether  or  not  integrating  the 
electronic  navigation  chart  and  radar/automated  radar  plotting  aid  (ARPA)  on  one 
device  would  contribute  to  navigation,  beyond  the  contributions  made  by  ECDIS 
used  with  a  separate  radar/ARPA,  has  been  debated.  Such  integration  might  reduce 
workload  by  reducing  the  number  of  devices  the  mariner  needed  to  consider  and 
might  improve  safety  by  improving  the  quality  of  information  available  to  the 
mariner.  The  capabilities  of  the  two  ECDIS  devices  allowed  experienced  mariners  to 
examine  two  implementation  possibilities:  the  overlay  of  a  complete  radar  video  on 
the  electronic  chart  or  the  addition  of  only  ARPA  targets  to  the  electronic  chart.  To 
determine  whether  the  integration  had  any  advantage,  performance  with  each 
integration  possibly  was  compared  to  performance  with  ECDIS  and  radar/ARPA  as 
separate  devices  on  the  bridge. 

3.2  EXPERIMENTAL  DESIGN 

Table  3-1  shows  the  experimental  conditions  in  three  groups:  with  the  Offshore  Systems 

Limited  (OSL)  device  available  on  the  bridge,  with  the  Robertson  Marine  Systems  Inc. 

device  available,  or  as  a  baseline  condition  with  no  ECDIS  available.  Each  ECDIS  system 

was  presented  in  three  different  modes: 

•  with  automatic  positioning  update  and  radar  overlay  to  allow  the  examination  of  the 
IMO  PPS  requirements  for  the  integration  of  radar  into  the  ECDIS  (Scenarios  1  and 
4) 

•  with  automatic  positioning  update  and  no  radar  overlay  as  the  baseline  configuration 
for  ECDIS  (Scenario  2,  5,  and  7) 


3-2 


.  with  ^ntnmatin  pnsitinninf  undate  to  allow  the  examination  of  the  IMO  PS  re¬ 
quirement  that  ECDIS  allow  the  user  to  perform  the  same  navigational  tasks  currently 
performed  with  the  traditional  paper  chart  (Scenario  3  and  6) . 

The  third  group  of  conditions  shown  in  Table  3-1  was  run  with  only  conventional  equip¬ 
ment  available  on  the  bridge,  to  provide  a  baseline  to  which  to  compare  the  more  novel 
ECDIS  conditions. 

Runs  took  place  in  two  “harbors.”  OSL  provided  electronic  charts  of  New  York  (NY)  in 
three  scales;  Robertson  provided  charts  in  two  scales  both  for  NY  and  for  San  Francisco. 
Routes  were  varied  to  include  coastal  and  harbor/harbor  approach  phases  of  navigation 
(Federal  Radionavigation  Plan,  1992)  and  to  include  inbound  and  outbound  transits.  Day 
and  night  and  different  visibilities  (clear,  unlimited  visibility,  and  reduced  visibility  to  1 
nautical  mile)  were  included  to  allow  varied  examination  of  ECDIS  and  its  radar  overlays. 

3.3  EXPERIMENTAL  SCENARIOS 

3.3.1  Experimental  Scenarios  and  the  Requirements  of  the  Evaluation 

The  scenarios  summarized  in  Table  3-1  were  designed  to  be  operationally  samyalsnLin 
terms  of  risk,  task  loading,  underlying  task  structure,  and  overall  mission  objectives.  At 
the  same  time,  the  scenarios  were  designed  to  be  different  in  internal  content  and  events. 
These  differences  served  several  purposes.  The  first  purpose  was  the  requirement  that 
ECDIS  be  evaluated  over  a  wide  range  of  realistic  operational  conditions  and  navigational 
regions.  Second,  differences  in  the  internal  content  of  the  scenarios  prevented  the 
mariner’s  anticipation  of  events  in  a  way  that  might  have  resulted  in  his  performance 
becoming  insensitive  to  the  experimental  variations  in  Table  3-1. 
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T able  3- 1 .  Experimental  Conditions 


SCENARIO 

HARBOR 

ROUTE 

VISBIUTY 

ECDIS 

POSITION 

RADAR/TARGET 

OVERLAY 

OFFSHORE  SYSTEMS  LTD  PINS-VME 

inbound: 

leduced 

automatic 

1 

NY 

coastal,  harbor 

visibility 

update 

yes 

inbound: 

clear/ 

automatic 

2 

NY 

coastal,  harbor 

day  -night 

ipdate 

no 

outbound: 

leduced 

manual/ 

3 

NY 

harbor 

visibility 

radar  overlay 

yes 

ROBERTSON  MARINE  SYSTEMS  INC  DISC  NAVIGATION 

inboun± 

reduced 

automatic 

4 

NY 

coastal 

visibility 

update 

yes 

inbound: 

clear/ 

automatic 

5 

SF 

coastal,  harbor 

day  -night 

update 

no 

inbound: 

clear/ 

manual/ 

6 

SF 

coastal 

day  -night 

conventional 

no 

outbound: 

reduced 

automatic 

7 

SF 

harbor,  coastal 

visibility 

update 

no 

CONVENTIONAL 

BRIDGE/PAPER 

CHART 

inbound: 

clear/ 

8 

NY 

coastal,  harbor 

day  -night 

NA 

NA 

inbound; 

clear/ 

9 

SF 

coastal,  harbor 

day  -night 

NA 

NA 

The  scenarios  were  designed  to  maintain  the  mariner’s  workload  at  a  relatively  high,  but 
sustainable,  level.  A  number  of  artificialities  were  introduced  into  the  scenario  procedures 
to  achieve  this  level.  Most  notably,  the  mariner  was  alone  on  the  bridge  (with  a  helmsman) 
during  conditions  when,  realistically,  there  would  be  at  least  one  other  officer. 
Additionally,  most  of  the  experimental  transits  went  from  the  coastal  confluence  zone  into 
harbor/pilotage  waters  without  taking  on  a  pilot.  The  expectation  was  that  the  high  work- 
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load  would  increase  the  sensitivity  of  the  performance  measures  to  differences  among 
scenario  conditions.  For  example,  with  the  high  workload  the  mariner  would  presumably 
be  forced  to  select  what  he  felt  to  be  the  most  effective  method  of  navigation  and,  possibly, 
drop  secondary  tasks  (e.g.,  bridge  management  tasks)  not  directly  related  to  the  ship’s 
safety.  If  the  workload  were  high  enough,  performance  in  the  primary  navigation  and 
collision  avoidance  functions  might  also  be  degraded. 

3.3.2  The.  F.xnerimental  Scenarios  and  the  Mariner’s  Tasks 

The  design  of  nine  unique  but  equivalent  scenarios  necessitated  the  development  of  a  con¬ 
ceptual  framework.  First,  the  tasks  and  functions  of  the  watchkeeper/conning  officer  in  the 
coastal  confluence  and  harbor/pilotage  areas  of  operation  were  identified  (Meum,  1990; 
Crenshaw,  1975).  These  tasks  and  functions  were  then  divided  into  three  broad  categories: 
navigation,  collision  avoidance  and  bridge  management  The  next  step  towards  scenario 
design  required  consolidating  the  tasks  and  functions  in  each  category  into  “generic”  events 
that  each  represented  an  operationally  logical  unit  Table  3-2  shows  the  generic  events 
which  were  used  to  construct  all  the  scenarios. 

Three  parallel  time  lines  of  component  events  for  navigation,  collision  avoidance  and  bridge 
management  events  where  planned  for  each  scenario.  An  experienced  mariner  then  created 
the  scenario  data  bases  by  placing  approximately  equal  numbers  of  generic  events  from 
each  category  into  each  scenario  to  form  operationally  logical  and  realistic  transits.  Since 
the  assessment  of  navigation  workload  and  performance  was  the  primary  focus  of  the  study 
and  since  collision  avoidance  was  intrinsically  related  to  safety  and  navigation,  the  intent 
was  to  keep  navigation  and  collision  avoidance  relatively  constant  throughout  a  scenario 
and  across  all  scenarios.  The  bridge  management  events/tasks  were  used  to  tune  each 
scenario  to  maintain  a  consistently  high  workload  during  slow  periods  or  to  maintain 
equivalence  when  a  particular  scenario  was  intrinsically  more  or  less  difficult  because  of  the 
nature  of  the  route.  The  design  of  the  experimental  scenarios  illustrates  the  type  of  control 
that  is  possible  on  a  simulator,  but  impossible  at  sea. 

The  equivalence  of  the  scenarios  was  examined  by  experienced  mariners  as  a  part  of  the 
preliminary  testing  during  the  experimental  preparation.  Revisions  were  made  to  some  of 
the  scenarios  to  increase  equivalence  before  the  principal  data  collection.  During  the  data 
analysis,  the  scenarios  were  again  examined  for  equivalency  by  comparing  the  mariners’ 
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Table  3-2.  Generic  Events  in  the  Development  of  the  Experimental  Scenarios 


NAVIGATION 

COLLISION 

AVOIDANCE 

BRIDGE 

MANAGEMENT 

-  precision  trackkeeping/ 
positioning 

-  own  ship  burdened, 
must  give  way 

-  non-time/location 
dependent  routine  task 

-  plan/calculate 

-  other  vessel  takes 
inappropriate  action 
(or  fails  to  take  action) 

time/location  dependent 
routine  task 

-  alter  course 

-  multi-ship  encounter, 
ownship  must  act 

-  externally  cued  task 

-  difficult  to  detect  target 

workload  ratings  across  scenarios.  The  scenarios  did  vary  in  navigation  workload  in  ways 
that  are  attributable  to  the  experimental  manipulations.  The  navigation  workload  data  are 
presented  in  Section  5.  There  were  some  unintended  differences  in  collision  avoidance 
workload  that  must  be  attributed  to  differences  in  the  difficulty  of  the  collision  avoidance 
events.  There  were  no  differences  among  the  scenarios  in  bridge  management  workload. 
These  incidental  data  on  collision  avoidance  and  bridge  management  are  presented  in 
Appendix  E. 

3 .4  INCLUSION  OF  THE  PARTICIPATING  MARINERS 

The  six  participants  for  the  study  included  four  Masters  (unlimited  license),  each  with  more 
than  20  years  of  experience,  one  Second  Mate  (two  years  of  experience),  and  one  Chief 
Mate  (nine  years  of  experience).  They  all  had  impressive  resumes  for  their  licenses  and 
had  simulator  and/or  computer  experience.  The  intention  was  to  select  mariners  who  could 
be  expected  to  adapt  to  the  new  technology  and  provide  good  performance  and  meaningful 
reactions  in  a  very  short  time.  The  mariners’  qualifications  and  experience  are  described  in 
greater  detail  in  Appendix  C. 


Each  mariner  spent  a  week  at  MSVCAORF,  becoming  familiar  with  both  ECDIS  systems, 
making  transits  through  aU  the  experimental  scenarios,  and  responding  to  extensive  post¬ 
scenario  and  post-experiment  questionnaires.  The  week’s  schedule  for  each  mariner  in¬ 
cluded  the  following  events: 
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1.  Training  with  the  first  ECDIS  device  (The  “first”  device  was  alternated  for  the  six 
mariners.) 

2 .  Familiarization  with  the  bridge  and  all  aspects  of  the  experiment 

3 .  Proficiency  testing  on  the  first  ECDIS  device 

4.  Experimental  scenarios  with  the  first  device  and  one  of  the  baseline  scenarios,  each 
followed  by  a  post-scenario  debriefing  (The  mariners  ran  through  the  scenarios  in  a 
different  order  so  that  position  in  the  week  did  not  bias  performance  in  any  one 
scenario.) 

5 .  Training  and  proficiency  testing  with  the  second  ECDIS  device 

6.  The  remaining  experimental  scenarios,  each  followed  by  a  post-scenario  debriefing 

7 .  An  extensive  post-experimental  debriefing 

These  events  are  described  in  greater  detail  in  Appendix  C. 
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4.0  PERFORMANCE  MEASURES 


Item  1  of  the  IMO  Standards  (International  Maritime  Organization,  1994)  suggests  that, 
compared  to  the  conventional  bridge,  ECDIS  should  provide  greater  or  equal  and 
should  reduce  workload.  In  order  to  test  these  hypotheses,  it  was  necessary  to  select 
performance  measures  to  represent  these  abstract  concepts.  In  addition,  measures  had  to  be 
selected  to  establish  preferences  for  specific  features  and  functions  on  the  ECDIS  systems. 
This  section  describes  the  selection  of  a  variety  of  measures  fiom  a  variety  of  sources. 

4.1  SAFETY 

The  measurement  of  “safety”  has  been  debated  in  shiphandUng  research  for  a  long  time. 
Generally,  the  solution  has  been  to  select  a  measure,  or  measures,  of  the  mariner’s  ability 
to  control  the  ship  (Gynther,  1985;  Kaufinan,  1985;  and  Schryver,  1985).  The  most 
commonly  used  of  these,  cross-track  distance  from  an  intended  track,  seems  especially 
appropriate  for  ECDIS  with  its  emphasis  on  a  geographical  presentation  of  the  navigation 
simation.  In  more  recent  years,  the  concept  of  “situational  awareness”  has  moved  from 
aviation  (Sarter  and  Woods,  1991;  Aretz,  1989;  Schwartz,  1989)  into  shiphandling  training 
and  has  been  adapted  by  MSI/CAORF  for  use  in  its  mariner  training  classes.  Situational 
awareness  was  included  here  as  a  measure  that  is  considered  closer  to  the  cognitive  activity 
of  the  human  operator  than  are  ship  tracks.  A  third  measure  based  on  the  assumption  that 
safety  is  related  to  performance  was  an  experimenter’s  rating  of  errors. 

4.1.1  Cross-n^ck  Distance  from  an  Intended  Track 

A  number  of  previous  simulation  studies  (Smith,  1993;  Smith  and  Mandler,  1992;  O'Hara 
and  Brown,  1985;  Aranow,  1979)  demonstrated  improved  trackkeeping,  better  execution 
of  turns  in  narrows  channels,  and  overall  improved  precision  of  navigation  with  the  provi¬ 
sion  of  ECDIS-like  devices.  These  findings  have  been  corroborated  at  sea  (Gonin  and 
Crowell,  1992;  Cooperand Bertsche,  1981). 

To  perform  the  analysis,  trajectory  plots  of  each  vessel’s  path  were  prepared  and  overlaid 
on  a  single  plot  for  each  scenario.  The  "composite  track  plot"  for  each  scenario  showed  the 
planned  trackline  for  the  scenario,  principal  points  of  reference  (buoys,  channel  lines,  cul¬ 
tural  features)  and  the  actual  route  followed  by  each  mariner  that  performed  the  scenario.  A 
full  set  of  these  plots  appears  in  Appendix  E. 
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In  the  experiment  reported  here,  there  were  no  instructions  for  mariners  to  keep  the  ship 
close  to  the  planned  track  as  there  had  been  in  some  earlier  studies.  They  were  permitted  to 
set  their  own  tolerance  for  distance  as  would  be  the  case  at  sea.  In  addition,  the  scenario 
design  required  them  to  leave  the  planned  track  in  response  to  frequent  traffic  encounters, 
some  of  them  quite  difficult.  However,  because  of  observed  performance  in  the  earlier 
studies,  it  was  hypothesized  that  the  precise  information  provided  by  ECDIS  would 
encourage  a  small  cross-track  distance,  even  without  instructions.  To  test  this  hypothesis, 
the  ship  tracks  from  the  present  experiment  were  examined  for  differences  between  ECDIS 
conditions  and  baseline  conditions  run  with  a  conventionally  equipped  bridge  without 
ECDIS.  Special  attention  was  given  to  critical  points  in  the  transits,  such  as  approaches  to 
major  turns,  precision  anchoring  maneuvers,  and  passage  under  bridges.  At  selected 
points  in  the  transit,  the  mean  cross-track  distances  from  the  intended  tracks  were  calculated 
for  both  the  ECDIS  and  conventional  bridge  conditions.  To  determine  whether  differences 
between  the  means  of  these  conditions  were  reliable  and  meaningful  a  statistical  procedure 
(the  "t"  test )  was  used.  A  summary  table  of  the  results  of  the  these  tests  appears  in 
Appendix  E.  Statistically  significant  (reliable)  effects  are  discussed  in  Section  5.0. 

4.1.2  Situational  Awareness 

Situational  awareness  is  defined  as  the  degree  to  which  there  is  an  accurate  perception  of 
the  factors  and  conditions  that  are  affecting  the  system  (Sarter  and  Woods,  1991).  The  un¬ 
derlying  premise  here  for  trying  to  assess  a  mariner’s  level  of  situational  awareness  at  any 
given  time,  is  that  there  is  a  positive  relationship  between  situational  awareness  and  current 
level  of  navigational  safety  in  the  waterway.  The  hypothesis  was  that  situational  awareness 
would  be  greater  with  ECDIS  on  the  bridge.  Because  there  are  no  established  techniques 
for  measuring  this  phenomenon  (although  see  Aretz,  1989),  the  experimental  staff 
developed  its  own  procedures  for  collection  and  analysis  for  the  purposes  of  this  ex¬ 
periment.  The  preliminary  analysis  suggested  an  effect  in  favor  of  the  ECDIS  conditions. 
Because  there  was  only  a  suggestion  of  an  effect,  a  decision  was  made  not  to  invest  further 
time  in  the  analysis.  The  procedures  that  were  used  and  the  preliminary  results  are  pre¬ 
sented  in  Appendix  D. 
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4.1.3  Fxperimenter’s  Rarings  of  Performance 


In  order  to  examine  a  broader  range  of  performance  and  the  “errors”  that  the  mariners  might 
make,  performance  during  the  experimental  runs  was  rated  by  an  observing  mariner  during 
each  scenario.  Structured  rating  scales  were  prepared  by  MSI/CAORF  mariners,  with 
separate  lists  of  possible  errors  for  the  navigation,  collision  avoidance,  and  bridge 
management  events  that  comprised  a  scenario.  The  hypothesis  was  that  fewer  errors  would 
be  committed  in  the  ECDIS  scenarios,  at  least  for  the  navigation  events.  Because  of  the 
relatively  exploratory  nature  of  this  measure  and  because  of  its  minimal  sensitivity  to  the 
experimental  manipulations,  formal  analyses  were  not  completed.  The  preliminary  findings 
are  presented  in  Appendix  D. 

4.2  WORKLOAD 

Workload  can  be  defined  as  the  amount  of  work  expected  from  an  individual  in  a  unit  of 
time,  using  the  resources  available  and  its  measurement  is  well  established  in  human  factors 
research  (Lysaght  et  al.,  1989).  Implicit  in  the  concept  of  ECDIS  is  the  assumption  that  the 
introduction  of  an  ECDIS,  providing  a  real-time  visual  display  of  ship’s  position  without 
the  need  for  plotting,  would  improve  safety  by  decreasing  operator  workload.  The 
assumption  is  that  safety  would  be  increased  because  the  watchstander  will  have  additional 
time  for  situational  evaluation  and  for  keeping  a  lookout.  A  number  of  workload  related 
measures  were  taken  in  an  attempt  to  capture  the  workload  experience. 

4.2.1  Mariner  Estimates  of  Time  Spent  Per  Task  Category 

As  mentioned  in  the  discussion  of  scenario  design  in  Section  3.3,  the  function  of  the  bridge 
watchstander  was  conceptualized  as  comprising  three  categories  of  tasks:  navigation, 
collision  avoidance,  and  bridge  management.  In  order  to  insure  that  mariners  would  be 
using  a  similar  understanding  of  the  kinds  of  activities  that  fall  into  each  category,  lists  for 
each  were  compiled  by  mariners  on  the  MSI/CAORF  staff  and  confirmed  by  consultants. 
These  lists  appear  in  Appendix  D.  With  these  lists  in  front  of  them,  the  experimental 
mariners  were  asked  to  estimate  the  proportion  of  time  they  had  spent  performing  tasks  in 
each  of  the  three  categories  during  the  transit  just  completed.  Data  obtained  for  ECDIS 
scenarios  were  compared  to  those  obtained  for  non-ECDIS  scenarios,  and  the  results  are 
presented  in  Section  5.0. 
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4.2.2  Mariner  Ratings  of  Workload  for  Task  Category 

After  a  review  of  the  literature,  documented  in  Phase  I  of  this  study  (MSI/CAORF  Staff, 
1991),  the  National  Aeronautic  and  Space  Administration’s  NASA  Task  Load  Index 
(NAS  A-TLX)  was  selected  as  a  measure  expected  to  be  sensitive  to  differences  in 
experimental  conditions  and  to  be  convenient  to  administer  and  to  analyze  (Hart  and 
Staveland,  1988).  The  mariner's  perception  of  workload  for  each  scenario  was  assessed 
using  this  scale.  After  the  mariner  had  estimated  the  proportion  of  time  spent  on 
navigation,  collision  avoidance,  and  bridge  management-related  tasks,  he  read  a  description 
of  the  six  NAS  A-TLX  workload  dimensions,  which  include  perceived  levels  of  mental, 
physical  and  temporal  demands,  own  performance,  effort,  and  frustration.  He  rated  his 
experience  of  workload  during  the  transit  on  a  zero  to  100  point  scale  on  each  of  the 
workload  dimensions,  thinking  only  about  the  navigation  tasks.  He  then  repeated  the 
workload  ratings  procedure  considering  only  the  collision  avoidance  tasks,  and  then  only 
the  bridge  management  tasks.  Each  scenario  produced  three  sets  of  workload  ratings  on 
each  of  the  six  scales.  For  each  category  of  bridge  task,  the  task-related  workload  score 
was  computed,  using  a  weighting  procedure  designed  by  the  NASA  developers.  Sample 
data  collection  materials  are  presented  in  Appendix  D. 

Reliability  of  differences  (statistical  significance)  among  workload  ratings  was  assessed  by 
a  Repeated  Measures  Analysis  of  Variance,  a  statistical  procedure  often  used  when 
measures  are  taken  of  the  same  individual  on  different  occasions.  In  this  case,  each 
mariner  provided  ratings  after  each  scenario.  Additional  statistical  tests  (single  degree  of 
freedom  contrasts)  were  used  to  determine  the  reliability  of  very  specific  comparisons:  for 
example,  whether  an  ECDIS  transit  was  superior  to  a  conventional  transit  in  the  same 
harbor  under  the  same  scenario  conditions.  In  selected  cases,  correlation  between 
measures  was  also  examined.  The  principal  findings  are  discussed  in  Section  5.0,  with 
some  supplementary  findings  presented  in  Appendix  E. 
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4.3  FEATURE  AND  FUNCTION  USE 


An  additional  issue  central  to  the  IMO  standards,  is  the  exanunation  of  which  specific  elec¬ 
tronic  chart  features  and  ECDIS-based  navigation  functions  are  useful  to  the  mariner.  A 
major  concern  in  this  study  was  the  usefulness  of  features  during  rpptg  monitoring.  The 
data  collected  included  direct  observation  of  the  mariners  ECDIS  use  during  transits  and 
their  responses  to  detailed  questionnaires.  The  various  methods  used  to  collect  information 
on  feature  and  function  use,  included: 

1 . )  experimenter's  tally  of  what  was  used  during  the  transit 

2.  )  the  mariner’ s  log  of  his  use  of  the  paper  chart 

3 .  )  post-scenario  questionnaires  on  features  used  or  features  needed  but  not  available 

4.  )  post-experiment  questionnaire  recording  the  mariner’s  recommendations  on  what 

should  be  available. 

Supplementary  descriptions  of  these  procedures  and  sample  materials  are  presented  in 
Appendbc  D.  A  detailed  presentation  of  the  resulting  data  can  be  found  in  Section  6.0. 

4.4  RADAR  INTEGRATION 

No  additional  performance  measures  were  used  to  evaluate  the  contribution  of  radar 
integration  to  ECDIS.  The  measures  described  above  were  used  to  evaluate  the  effects  of 
the  radar  integration  on  safety  and  workload  and  to  examine  the  features  used  and 
preferred. 
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5.0  RESULTS  AND  DISCUSSION:  CONTRIBUTION  OF  ECDIS  TO 
SAFETY  AND  WORKLOAD  IN  ROUTE  MONITORING 

5. 1  MARINERS’  VIEWS  OF  ECDIS  FOR  ROUTE  MONITORING 

5.1.1  Mariners’  Reports  of  the  Primary  Method  Used  for  Navigation 

To  provide  a  context  for  performance  in  each  experimental  scenario,  immediately  after  the 
scenario,  the  mariner  was  asked  to  report  the  primary  method  that  he  had  used  for 
navigation  in  each  identifiable  segment  of  the  transit.  The  possible  methods  were  paper 
chart,  radar/ARPA,  visual  piloting,  and  ECDIS.  Their  reports  are  summarized  in  Table  5- 
1.  The  reports  are  divided  into  coastal  and  harbor/harbor  approach  phases  (Federal 
Radionavigation  Plan,  1992)  with  the  change-over  at  the  pilot  station,  on  the  assumption 
that  the  navigation  problem  changes  at  that  point.  The  table  shows  the  proportion  of  times 
that  the  mariner  reported  that  a  particular  bridge  “tool”  was  used  as  the  primary  navigation 
method  during  each  bridge  condition:  the  conventionally  equipped  bridge,  ECDIS  with 
automatic  updating  of  position,  and  ECDIS  without  automatic  updating  of  position. 

Under  the  demanding  experimental  conditions,  the  one  mariner  alone  on  the  bridge  made 
use  of  time-consuming  plotting  on  the  paper  chart  only  in  the  coastal  phase  of  navigation 
and  only  with  the  conventionally  equipped  bridge.  It  is  for  this  reason  that  this  report 
refers  to  the  baseline  scenarios,  not  as  the  "paper  chart"  but  more  generally,  as  the 
"conventionally  equipped  bridge"  or  as  "conventional  methods."  Radar  use  was  reported 
for  navigation  with  some  frequency  in  all  conditions,  whatever  the  visibility  and  whether  or 
not  ECDIS  was  available.  This  use  was  in  addition  to  its  obvious  use  for  collision 
avoidance.  Presumably,  this  use  of  radar  for  navigation  was  a  by-product  of  the  need  to 
refer  to  it  frequently  in  all  conditions  for  collision  avoidance.  Visual  piloting  was  used  in 
the  harbor/harbor  approach  phase,  as  would  be  expected  (Crenshaw,  1975),  but  not  in  the 
coastal  phase.  F.CDTS  with  automatic  positioning,  when  it  was  available,  reduced  the  use 
of  all  other  methods  of  navigation  and  became  dominant.  In  the  coastal  phase,  the 
difference  between  the  radar  and  the  available  ECDIS  was  large  and  reliable  (statistically 
significant,  with  F  =  23.23,  p  <  0.(X)9);  in  the  harbor/harbor  approach  phase,  the 
difference  between  the  second-most-preferred-method,  visual  piloting,  and  ECDIS  was 


5-1 


Table  5- 1 .  Primary  Method  of  Navigation  as  Reported  by  Mariners 


PROPORTION  OF  SEGMENTS  USING  EACH  METHOD 
IN  EACH  BRIDGE  CONDITION 

BRIDGE 

CONDITIONS 

PLOTTING/ 

Paper 

CHART 

RADAR/ 

ARPA 

VISUAL 

PILOTING 

ECDIS 

TOTAL 

NUMBER 

SEGMENTS 

COASTAL  PHASE  OF  NAVIGATION 

Conventional 

Bridge 

0.43 

0.57 

0.00 

NA 

14 

ECDIS  Auto 
Positioning 

0.00 

0.17 

0.00 

0.83 

24 

ECDIS  no  Auto 
Positioning 

0.29 

0.71 

0.00 

0.00 

7 

HARBOR/HARBOR  APPROACH  PHASE  OF  NAVIGATION 

Conventional 

Bridge 

0.03 

0.25 

0.73 

NA 

40 

ECDIS  Auto 
Positioning 

0.00 

0.15 

0.18 

0.67 

79 

ECDIS  no  Auto 
Positioning 

0.05 

0.61 

0.28 

0.05 

18 

also  reliable  or  statistically  significant  (F  =  15.96,  p  <  0.02).  Alone  on  the  bridge  and  with 
a  high  workload,  the  mariners  preferred  ECDIS  as  their  method  of  navigation.  ECDIS 
without  automatic  positioning  was  used  very  seldom  as  the  primary  method.  With  a  failure 
of  electronic  positioning  and  with  instructions  to  manually  update  the  ship's  DR  position, 
the  mariners  frequently  made  use  of  the  radar/ARPA  to  obtain  a  position  and  transferred 
that  position  to  the  ECDIS. 
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5.1.2  Mariners’  Acceptance  of  ECDIS  for  Navigation 


The  comments  made  by  the  mariners  during  de-briefings  provide  an  indication  of  how  they 
saw  the  ECDIS  contributing  to,  or  inhibiting,  their  actions.  The  most  commonly 
mentioned  positive  a.sDects  of  ECDIS  use  during  high-stress  events  was  that  it  provided 
quick  and  continuous  confidence  in  own  ship's  position  relative  to  charted  objects  such  as 
channels  and  bridges.  Many  of  the  mariners  mentioned  that  navigational  workload  was 
eased,  allowing  them  to  concentrate  on  collision  avoidance.  They  also  mentioned, 
however,  that  ECDIS  did  not  help  with  collision  avoidance  during  these  events. 

Among  the  more  negative  aspects  of  ECDIS  use  during  high-stress  events  was  a  height¬ 
ened  attention  to  own  ship’s  distance  from  the  planned  trackline  that  interfered  on  occasion 
with  the  task  of  watching  for  traffic.  (There  had  been  no  explicit  instractions  to  keep  the 
ship  close  to  that  planned  track.)  This  concern  with  the  cross-track  distance,  which  was 
obviously  apparent  on  the  ECDIS,  resulted  in  a  near  miss  with  a  another  ship  in  a  couple  of 
cases,  leading  to  increased  anxiety.  The  mariners  also  lost  confidence  in  ECDIS 
positioning  on  occasion,  due  to  inconsistencies  of  object  locations  between  the  paper  and 
electronic  charts.  (These  electronic  charts  were  not  from  National  Oceanic  and 
Atmospheric  Administration,  NOAA.)  Finally,  some  mariners  mentioned  that  at  times  they 
were  not  able  to  operate  the  ECDIS  to  obtain  the  information  they  required,  causing  them  to 
revert  to  conventional  methods  of  navigation  when  under  pressure.  Presumably,  they  felt 
that  that  was  the  most  effective  use  of  limited  time. 

5.2  CONTRIBUTION  OF  ECDIS  TO  THE  SAFETY  OF  NAVIGATION 

5.2.1  Effects  of  ECDIS  on  Cross-Track  Distance  from  an  Intended  Track 

Composite  track  plots,  as  described  in  Section  4.1,  were  examined  for  differences  at¬ 
tributable  to  the  presence  or  absence  of  ECDIS  among  the  scenarios.  A  summary  data  table 
and  a  full  set  of  composite  track  plots  appear  in  Appendix  E. 

The  participating  mariners  reported  that  when  precision  navigation  was  required,  as  in  the 
harhnr/pilotage  regions.  ECDIS  was  their  primary  method  of  navigation.  Track  plots  of 
mariner/ship  performance  were  examined  for  a  decrease  in  cross-track  distance  with 
ECDIS.  The  most  conspicuous  examples  are  the  greater  precision  found  with  ECDIS  runs 
in  San  Francisco  Harbor.  Figure  5-1  compares  Plot  5c  (Scenario  5)  and  conventional 
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Figure  5-1. 


Composite  Track  Plots  Using  ECDIS  (Scenario  5)  and  Using  Conventional 
Bridge  Procedures  (Scenario  9) 
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bridge  conditions  in  Plot  9c  (Scenario  9).  Significant  mean  cross-track  distances  were 
found  there  as  summarized  in  Table  5-2.  A  similar  trend  of  decreased  cross-track  distance 
with  ECDIS  was  observed  in  New  York  Harbor,  as  illustrated  in  Appendix  E  in  Plots  2b 
and  2c  (Scenario  2)  compared  to  Plots  8b  and  8c  (Scenario  8).  Also  apparent  in  the  New 
York  plots  is  a  greater  precision  with  ECDIS  in  the  anchoring  maneuver,  although  no  sta¬ 
tistical  significance  was  found  in  the  cross-track  distances  measured  for  the  New  York  sce¬ 
narios. 


Table  5-  2.  Mean  Cross-Track  Distance  with  and  Avithout  ECDIS 


LOCATION 

MEAN  Cross-track 
DISTANCE  (METERS) 

Statistical 

SIGNMCANCE 

SAN  FRANCISCO 

ECDIS 
Auto 
Position 
(Scenario  5) 

Conventional 
Bridge 
(Scenario  9) 

Golden  Gate  Bridge 

32 

117 

0.05 

Alcatraz  1st  turn 

18 

100 

0.09 

Alcatraz  2nd  turn 

29 

98 

0.06 

While  mariners  reported  that  ECDIS,  when  it  was  available,  was  also  the  primary  method 
of  navigation  in  the  coastal  region,  trackkeeping  performance  in  those  scenarios  showed 
no  improvement  over  performance  in  the  conventionally-equipped  bridge  scenarios. 
Presumably,  there,  the  mariners  considered  it  less  critical  to  stay  close  to  the  planned 
trackline  and  did  not  take  advantage  of  the  accurate  position  information  that  ECDIS 
offered. 

There  was  evidence,  however,  that  using  ECDIS,  individual  mariners  made  wide 
departures  from  the  trackline  in  response  to  traffic  or  to  avoid  a  developing  traffic 
encounter.  Figure  5-2  presents  a  striking  example,  showing  that  with  ECDIS  in  Plot  2a 
(Scenario  2),  there  is  an  extreme  excursion  unlike  any  track  made  without  ECDIS  in  Plot  8a 
(Scenario  8).  Such  departures  with  ECDIS  can  also  be  seen  in  Appendix  E  in  Plot  5a 
(Scenario  5)  as  compared  to  the  conventional  condition  in  Plot  9a  (Scenario  9).  No 
statistical  tests  were  done  in  the  coastal  regions  because  planned  routes  were  different 
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ECDIS  with  Auto-Update  but  No  Radar  Overlay, 
— I  Inbound  NY  in  Clear  Visibility  (seen.  2) 


Figure  5-2.  Composite  Track  Plots  Using  ECDIS  (Scenario  2)  and  Using  Conventional 

Bridge  Procedures  (Scenario  8) 
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between  the  scenarios  (These  differences  had  been  designed  to  prevent  the  mariners  from 
anticipating  events  from  one  transit  to  the  next.).  Such  patterns  of  response  to  traffic  were 
also  seen  in  a  previous  study  that  found  improved  and  more  decisive  collision  avoidance 
maneuvers  with  addition  of  ECDIS-like  capability  to  an  ARPA  (Hayes,  1979). 

5.3  EFFECT  OF  ECDIS  ON  THE  WORKLOAD  OF  THE  TRANSIT 

5  3  1  Fffcr-t  of  Navigation  Workload  on  the  Distribution  of  Time  as  Reported  bv  th£ 
Mariners 


After  each  scenario  the  mariners  were  asked  to  rate  their  perceived  workload,  using  the 
NASA  Task  Load  Index  (NASA-TLX)  described  in  Section  4.2.  Separate  ratings  were 
given  for  navigation,  collision  avoidance,  and  bridge  management  tasks.  The  mean  work¬ 
load  rating  for  navigation  is  presented  in  Table  5-3.  Table  5-3  shows  that  the  availability  of 
ECDIS  with  automatic  positioning  decreased  navigation  workload  from  conventional  pro¬ 
cedures  and  that  attempts  to  use  ECDIS  without  automatic  positioning  increased  workload. 
These  differences  in  workload  were  statistically  different  (F  =  24.69,  p  <  0.006  and  F  = 
41.89,  p  <  0.002,  respectively).  The  use  of  ECDIS  without  auto-positioning  is  discussed 
further  in  Section  5.4. 

To  explore  further  the  nature  of  mariner’s  workload,  distribution  of  time  was  also  exam¬ 
ined.  After  each  scenario  the  mariners  were  asked  to  estimate  the  proportion  of  total  time 
spent  on  each  category  of  activity:  navigation,  collision  avoidance,  and  bridge 
management.  The  means  of  reported  proportions  are  summarized  in  Table  5-3.  Statistical 
tests  showed  that  the  decrease  in  proportion  of  time  spent  on  navigation  from  the  conven¬ 
tional  bridge  to  ECDIS  with  automatic  positioning  was  significant  (F  =  5.61,  p  <  0.06). 
The  corresponding  increase  in  proportion  of  time  spent  on  collision  avoidance  with  ECDIS 
with  automatic  positioning  was  also  significant  (F  =  5.04,  p  <  0.07).  Differences  between 
the  ECDIS  with  n2  automatic  positioning  and  the  other  bridge  conditions  were  not 
statistically  significant.  This  reciprocal  relation  between  the  proportions  of  time  spent  on 
navigation  and  on  collision  avoidance  is  illustrated  in  Figure  5-3. 
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Table  5-3. 


Navigation  Workload  and  Reported  Distribution  of  Mariner’s  Time 


MEAN  PROPORTION  OF  TIME  ON  TASK 
IN  EACH  BRIDGE  CONDITION 

BRIDGE 

CONDITIONS 

MEAN 

NAVIGATION 

WORKLOAD 

Navigation 

COLLISION 

Avoidance 

Bridge 

Management 

Conventional 

Bridge 

52 

0.46 

0.33 

0.21 

ECDIS  Auto 
Positioning 

36 

0.37 

0.41 

0.21 

ECDIS  no  Auto 
Positioning 

63 

0.49 

0.34 

0.17 

Note:  Proportions  that  do  not  total  to  1.0  are  the  result  of  rounding  error. 


Figure  5-3.  Mean  proportion  of  time  spent  on  bridge  tasks  during  conventional  bridge 
scenarios  versus  ECDIS  scenarios 


Note  that  the  proportion  of  time  spent  on  bridge  management  tasks,  as  reported  in  Table  5- 
3,  does  not  appear  to  be  affected  by  the  presence  or  absence  of  an  ECDIS  on  the  bridge. 
This  finding  is  consistent  with  comments  made  by  some  of  the  mariners  that  having  an 
ECDIS  on  board  as  a  navigation  device  does  not  really  reduce  or  make  easier  the  bridge 
management-related  tasks.  That  is,  the  mariners  still  had  to  fiU  out  communications  log 
sheets,  make  bell  book  entries,  perform  and  record  gear  tests,  etc. 
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Since  the  addition  of  ECDIS  with  automatic  positioning  to  the  bridge  was  observed  to  af¬ 
fect  two  performance  measures  -  the  perceived  mariner  workload  and  the  proportion  of 
time  spent  on  a  category  of  task  --  correlations  between  them  were  examined.  Workload 
for  navigation  and  the  proportion  of  time  spent  on  navigation  were  positively  and  signifi¬ 
cantly  correlated  (r  =  0.51,  p  <  0.002).  Decreased  workload  for  navigation  was  associated 
with  decreased  proportion  of  time  spent  in  navigation.  Decreased  workload  for  navigation 
was  also  correlated  with  an  increase  in  the  proportion  of  time  spent  on  the  collision 
avoidance  task  (r  =  -  0.575,  p  <  0.002).  To  summarize  the  findings,  ECDIS  with 
automatic  positioning  was  associated  with  a  decreased  workload  for  navigation,  a 
decreased  proportion  of  time  spent  on  navigation,  and  an  increased  proportion  of  time  spent 
on  the  collision  avoidance  task. 

An  impUcit  hypothesis  underlying  Item  1  of  the  IMO  standards  (International  Maritime 
Organization,  1994)  is  that  there  is  a  relation  between  decreased  workload  and  increased 
safety.  These  findings  support  such  a  relation  and  suggest  a  mechanism.  With  the 
introduction  of  ECDIS  to  the  bridge  the  mariner  is  relieved  of  the  necessity  of  manually 
fixing  his  position  every  few  minutes,  and  thereby  has  more  time  to  spend  on  other  tasks. 
The  mariners  in  this  experiment  spent  the  savings  in  time  on  “look  out”  and  on  the  collision 
avoidance  task.  This  extra  attention  to  collision  avoidance  suggests  a  potentially  greater 
level  of  safety  in  the  waterway  when  an  ECDIS  is  on  board. 

5.3.2  Pp-rcdved  Workload  for  Navigation.  Collision  Avoidance,  and  Bridge  Management 

After  each  scenario  the  mariner  was  asked  to  rate  separately  his  experience  of  navigation, 
collision  avoidance,  and  bridge  management  workload,  using  the  NASA-TLX  workload 
measurement  technique  described  in  Section  4.2.  Overall,  this  technique  worked  well  in 
discriminating  among  perceived  workload  levels  produced  by  the  different  scenarios. 


The  mean  navigation,  collision  avoidance,  and  bridge  management  workload  scores  are 
plotted  in  Figure  5-4  for  the  conventional  bridge  scenarios  as  compared  to  the  ECDIS-with- 
automatic-positioning  scenarios.  As  can  be  seen,  the  reduction  in  perceived  navigation 
workload  as  a  result  of  the  availability  of  ECDIS  is  substantial.  An  analysis  of  variance  on 
these  data  and  a  subsequent  single  degree  of  freedom  contrast  show  that  the  difference  in 
mean  workload  for  navigation  is  highly  significant  (F  =  24.69,  p  <  0.006).  The  collision 
avoidance  effect  shown  here  is  also  significant,  as  an  artifact  of  differences  among 
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Figure  5-4.  Mean  workload  for  navigation,  collision  avoidance,  and  bridge  management 
tasks  during  conventiond-bridge  (Sc  8  &  9)  versus  ECDIS  scenarios  (Sc  1, 
2,  4,  5,  &  7)  scenarios. 


The  bridge  management  effects  is  not  significant.  Further  analyses  of  the  workload  data 
are  presented  in  Appendix  E. 

5 .4  EFFECTS  OF  USING  ECDIS  WITH  A  LOSS  OF  AUTOMATIC  POSITIONING 

5.4.1.  Effects  of  Loss  of  Automatic  Positioning  on  the  Cross-Track  Distance 

The  mariners  experienced  each  of  the  two  ECDIS  devices  without  their  automatic  DGPS 
positioning  feature  and  with  instructions  to  manually  update  the  DR  position.  The  track 
plots  were  examined  for  the  possible  effects  on  cross-track  distance.  Generally,  this 
measure  was  not  affected  in  the  coastal  regions.  However,  it  was  affected  in  the 
harbor/pilotage  regions.  Comparisons  were  made  between  ECDIS  in  the  auto-update  mode 
(Scenarios  1  and  5)  and  ECDIS  in  the  manual  update  mode  (Scenarios  3  and  6).  The  most 
extreme  effect  is  in  the  comparison  between  New  York  Scenarios  1  and  3  illustrated  in 
Plots  lb  and  3b,  in  Figure  5-5.  The  figure  shows  that,  while  trajectories  in  Ambrose 
Channel  meander  about  the  planned  trackline  in  both  scenarios,  trajectories  are  much  tighter 
around  the  first  and  second  turns  when  ECDIS  is  automatically  updating  the  position  of 
own  ship  (Scenario  1).  The  mean  cross-track  distances  in  the  turns  are  summarized  in 
Table  5-4.  T- tests  on  these  data  show  significant  differences  at  the  first  turn  (t  =  2.925,  p 
<  0.09)  and  the  second  turn  (t  =  12.39,  p  <  0.006). 

Comparisons  between  San  Francisco  Scenarios  6  and  5,  illustrated  in  Appendix  E,  show  a 
similar  effect.  With  automatic  updating,  trajectories  clustered  more  tightly  around  the 
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Figure  5-5.  Composite  Track  Plots  Using  ECDIS  with  Automatic  Positioning  (Scenario 
1)  and  Using  ECDIS  with  Manual  Updating  (Scenario  3) 


Table  5-4. 


Mean  Cross-Track  Distance  with  and  without  Automatic 
Updating  of  Position 


lxx:ation 

MEAN  Cross-track 
Distance  (METERS) 

PROBABILITY 

NEW  YORK 

ECDIS 
Auto 
Position 
(Scenario  1) 

ECDIS 

No  Auto 
(Scenario  3) 

Ambrose  1st  turn 

41 

128 

0.09 

Ambrose  2nd  turn 

59 

219 

0.006 

Verrazano  Bridge 

18 

148 

0.18 

planned  trackline,  both  at  the  entrance  to  San  Francisco  Channel  and  through  the  Channel 
to  the  Golden  Gate  Bridge.  However,  differences  at  these  points  were  not  statistically  sig¬ 
nificant. 

During  the  test  runs,  the  mariners  were  frequently  observed  to  abandon  the  manual  updat¬ 
ing  of  ECDIS  opting  instead  for  conventional  procedures,  apparently  feeling  they  could  not 
safely  navigate  with  ECDIS  in  this  mode.  To  determine  whether  this  objective  measure  of 
safety,  cross-track  distance,  verified  their  judgment,  trackkeeping  performance  using 
ECDIS  in  the  manual  update  mode  was  compared  to  that  using  conventional  bridge  proce¬ 
dures.  Trackkeeping  performance  in  coastal  regions  was  not  different  between  these  two 
conditions.  However,  trackkeeping  performance  in  the  harbor/pilotage  regions  was 
poorer,  when  using  ECDIS  with  manual  updating  of  position,  than  that  observed  in  the 
baseline  conventional  navigation  scenarios. 

Performance  with  manual  updating  in  New  York  Scenario  3,  illustrated  in  Plot  3b  of  Figure 
5-5,  shows  that  the  trajectories  were  clustered  more  loosely  around  both  turns  in  Ambrose 
Channel  than  was  the  case  in  the  comparable  conventional  bridge  Scenario  8,  illustrated  in 
Plot  8b  in  Figure  5-6.  Mariners  were  consistently  further  off-track  when  manually  updat¬ 
ing  their  position  with  ECDIS  than  with  conventional  navigation.  T-tests  performed  on  the 
cross-track  data  showed  a  trend  toward  such  a  difference,  at  both  the  first  turn  in  Ambrose 
Channel  (t  =  2.053,  p  <  0.17)  and  the  second  turn  (t  =  2.176,  p  <  0.16).  A  similar  pattern 
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5-6.  Composite  Track  Plots  Using  Conventional  Bridge  Procedures  (Scenario  8) 


was  observed  in  the  track  plots  for  corresponding  San  Francisco  Scenarios  6  and  9  which 
appear  in  Appendix  E.  Mariners  reported  they  had  greater  difficulty  making  the  turn 
around  the  San  Francisco  Buoy  (Plot  6a)  when  manually  updating  ECDIS.  However,  t- 
tests  revealed  no  significant  differences  in  cross-track  distance  between  this  scenario  and 
conventional  navigation  in  Scenario  9.  Overall,  cross-track  distances  suffered  in  the 
manual  update  scenarios. 

5.4.2  Effects  of  Lx)ss  of  Automatic  Positioning  on  the  Mariners’  Workload 

Figure  5-7  shows  the  mean  proportion  of  time  spent  on  navigation-related  tasks  during  the 
OSL  and  Robertson  manual  update  scenarios  (Scenarios  3  and  6,  respectively)  as  compared 
to  each  device  in  its  normal  operating  mode  (Scenarios  1  and  5).  The  increase  in  the  pro¬ 
portion  of  time  spent  on  navigation  is  large,  especially  for  the  OSL  device. 


Auto  Manual  w/Auto  Manual 

Update)  Update)  Update)  Update) 


Figure  5-7.  Mean  proportion  of  time  spent  on  navigation  with  automatic  versus  manual 
updating  of  position 

Figure  5-8  shows  the  mean  navigation  workload  for  the  scenarios  with  automatic  updating 
of  position  compared  to  those  without  it  (Scenarios  1  versus  3  and  5  versus  6).  Navigation 
workload  was  dramatically  increased  when  the  mariner  was  required  to  manually  update  his 
position  on  the  ECDIS,  as  compared  to  when  the  ECDIS  had  automatic  DGPS  positioning. 
The  increase  in  mean  navigation  workload  was  statistically  significant  comparing  Scenarios 
1  and  3  (F  =  20.70,  p  <  0.008)  and  comparing  Scenarios  5  and  6  (F  =  21.20,  p  <  0.008). 
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Figure  5-8 .  Mean  navigation  workload  with  automatic  versus  manual  updating  of 
position 

The  routes  for  these  transits  are  illustrated  in  Appendix  E,  in  the  composite  track  plots. 

Note  that  the  transit  routes  being  compared  in  Figures  5-7  and  5-8  are  not  of  equal  length. 
WhUe  Scenarios  1  and  3  were  both  New  York  transits  in  reduced  visibility.  Scenario  3  was 
outbound  and  only  ran  from  Stapleton  anchorage  to  the  pilot  station.  Scenario  1,  on  the 
other  hand,  ran  all  the  way  from  the  coast  into  Stapleton  Anchorage  -  a  much  longer  transit. 
The  difference  in  mean  navigation  workload  is  even  more  impressive  considering  the  extra 
distance  transited  in  Scenario  1,  suggesting  that  the  manual  updating  of  the  ECDIS  may 
have  affected  the  navigation  workload  substantially.  A  similar  argument  could  be  made 
comparing  the  transit  routes  for  Scenarios  5  and  6.  Both  were  San  Francisco  transits  in 
clear  visibility,  but  Scenario  5  went  all  the  way  from  the  coast  into  Anchorage  9  while 
Scenario  6,  the  manual  update  scenario,  stopped  at  the  Golden  Gate  Bridge. 
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The  conventional  bridge  transits  (made  without  an  ECDIS)  discussed  in  Section  5.2.1 
above  were  also  relatively  long  transits,  extending  from  the  coast  all  the  way  to  Stapleton 
Anchorage  (Scenario  8)  or  Anchorage  9  (Scenario  9).  Even  so,  the  mean  navigation 
workload  was  perceived  to  be  greater  in  the  manual  update  scenarios  than  for  the  conven¬ 
tional  bridge  scenarios  (Scenarios  3  vs.  8:  F  =  5.77,  p  <  0.06;  Scenarios  6  vs.  9:  F  = 
2.66,  p  <  0.12).  To  summarize,  ECDIS  without  automatic  updating  of  position,  and  the 
requirement  to  manually  update  the  position,  resulted  in  a  significantly  higher  workload 
than  either  ECDIS  with  automatic  positioning  or  conventional  procedures. 
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The  extremely  negative  view  of  ECDIS  without  automatic  updating  of  position  should  be 
qualified,  however  there  was  only  one  mariner  on  the  bridge  with  other  responsibilities, 
the  mariners  were  instructed  to  update  the  position  on  EQ)IS  in  pilotage  waters  where  they 
ordinarily  would  not  be  alone  on  the  bridge  plotting  on  the  paper  chart,  both  of  the  ECDIS 
systems  used  were  prototypes  with  relatively  tedious  procedures  for  manually  updating, 
and  these  mariners  had  relatively  little  experience  with  the  systems.  Under  some 
circumstances,  manual  updating  might  be  more  effective  than  it  was  seen  to  be  here. 
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6.0  RESULTS  AND  DISCUSSION;  ROLE  OF  ECDIS  FEATURES  IN  ROUTE 
MONITORING 

This  section  presents  the  role  of  the  chart  features  and  navigation  functions  that  were  avail¬ 
able  on  the  ECDIS  systems.  The  characteristics  of  the  Robertson  and  Offshore  Systems 
Limited  (OSL)  systems  are  described  briefly  in  Section  2.2  and  in  greater  detail  in  Appendix 
B.  The  methods  of  data  collection  are  described  in  Section  4.3  and  in  Appendix  D.  Also 
reported  in  this  section  are  the  additions  or  modifications  that  these  mariners,  given  their 
simulator  experience  navigating  with  ECDIS,  would  have  liked  to  have  seen  and  their  views 
on  what  information  should  be  permanently  or  optionally  displayed.  Remember  that  the 
mariners  experienced  the  systems  only  for  route  monitoring.  Their  preferences  may  have 
been  different  for  route  planning. 

6. 1  OBSERVATIONS  ON  THE  NEED  FOR  DISPLAY  ACCURACY 

Earlier  research  (Smith  and  Mandler,  1992)  has  shown  that  a  high  level  of  display  accuracy 
is  necessary  for  safety  of  navigation  and  for  mariner  confidence.  Therefore,  the  experi¬ 
menters  were  very  sensitive  to  the  accuracy  of  the  displayed  information.  First,  a  variety  of 
problems  were  found  with  the  digitizing  accuracy  of  the  electronic  charts  received  from  the 
vendors  of  the  two  commercial  ECDIS  systems  used.  This  was  especially  tree  with  elec¬ 
tronic  charts  digitized  from  small  scale  paper  charts.  These  errors  --  in  one  case  a  buoy  was 
250  feet  out  of  place  -  had  major  effects  on  the  mariner's  trust  in  the  systems  and  have  ob¬ 
vious  implications  for  safety  of  navigation.  These  observations  support  a  conclusion  that 
electronic  charts  must  be  based  on  electronic  navigation  charts  (ENCs)  developed  by  na¬ 
tional  hydrographic  offices  in  conformity  with  international  standards.  Certainly,  the  pos¬ 
sibility  that  ECDIS  systems  would  enable  mariners  to  digitize  their  own  charts  should  be 
approached  with  great  caution. 

The  simulator  data  bases  were  modeled  with  the  high  levels  of  accuracy  that  are  character¬ 
istic  of  DGPS  for  several  reasons  including:  the  requirement  in  the  Federal  Radionavigation 
Plan  (Federal  Radionavigation  Plan,  1992)  of  a  high  level  of  positioning  accuracy  for  the 
harbor/harbor  approach  phase  of  navigation,  earlier  research  that  supported  a  need  for  highly 
accurate  positioning  for  piloting  in  restricted  waterways  (Smith  and  Mandler,  1992;  Gynther 
and  Smith,  1989),  and  the  present  availability  of  DGPS.  The  observed  concern  of  the 
mariners  with  the  accuracy  and  credibility  of  the  display  suggests  that  ECDIS  should  be 
used  in  the  harbor/harbor  approach  phases  of  navigation  only  with  high  levels  of  positioning 
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accuracy  (whether  that  accuracy  is  achieved  using  DGPS  or  some  other  means).. 
Participating  mariners  pointed  out  that,  in  other  circumstances,  ECDIS  would  be  a  useful 
addition  to  the  bridge  as  long  as  the  navigator  can  achieve  positioning  accuracy  equal  to  or 
better  than  that  currently  available  to  him  with  conventional  methods,  with  equal  or  less 
effort  than  currently  required. 

The  mariners'  concern  for  display  accuracy  included  the  scaled  ship  image  presented  on 
larger  chart  scales.  One  of  the  devices  presented  a  scaled  image  at  the  larger  chart  scales; 
the  other  presented  a  ship  image  that  was  not  scaled  for  the  dimensions  of  own  ship  or  for 
the  chart  scale.  There  was  a  strong  preference  for  the  accurately  scaled  outline.  This  pref¬ 
erence  is  consistent  with  earlier  fmdings  of  the  best  performance  and  confidence  for  piloting 
in  restricted  waterways  with  a  scaled  ship  image  against  channel  outlines  (Smith,  1993; 
Smith  and  Mandler,  1992;  Gynther  and  Smith,  1989).  Indeed,  in  one  study  (Smith,  1993) 
marine  pilots  suggested  a  scaled  outline  for  traffic  ships  to  be  met  in  narrow  channels. 

6.2  USE  OF  ELECTRONIC  CHART  SCALES 

One  of  the  most  frequently-observed  mariner  actions  was  the  changing  of  chart  scales.  The 
two  systems  differed  both  in  the  size  of  the  chart  display  and  in  their  treatment  of  chart 
scales  and,  therefore,  need  to  be  described  separately.  The  Robertson  system  had  a  dedi¬ 
cated  63.5-centimeter  (cm)  (25-inch)  screen  with  horizontal  and  vertical  dimensions  of 
52.83  cm  by  35.31  cm  (20.8  inches  by  13.9  inches).  The  chart  scales  were  labeled,  as  they 
are  for  paper  charts,  as  1:  20,(X)0,  etc.  The  scales  that  were  chosen  through  the  menus  are 
listed  in  the  first  column  in  Figure  6-1.  The  second  column  in  the  figme  indicates  the  size 
of  the  geographic  area  that  was  visible  on  the  single  Robertson  screen  for  those  scales  that 
were  most  frequently  chosen  by  the  mariners.  The  frequencies  with  which  each  chart  scale 
was  chosen  is  presented  in  the  third  column.  In  general,  the  mariners  tended  to 
"experiment"  with  the  chart  at  different  scales  (accounting  for  the  low  frequencies  for  many 
of  the  chart  scales)  but  usually  came  back  to  the  preferred  scales  indicated  by  the  higher 
frequencies.  For  the  Robertson  system,  the  most  preferred  scales  were  1:  20,000,  1:  50,000, 
and  1:100,000,  the  harbor  and  coastal  scales  that  were  most  familiar  from  paper  chart  use. 

The  OSL  system  had  a  48.26-cm  (19-inch)  screen  that  could  be  configured  to  display  a 
number  of  windows.  The  most  frequently  used  configuration  had  a  larger,  main  chart  to  the 
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Figure  6-1 .  Tallies  of  the  Frequency  of  Chart  Scale  Selection 


left  and  a  smaller,  second  chart  in  the  upper  right  hand  comer.  With  this  configuration,  the 
main  chart  was  26.67  cm  square.(10.5  by  10.5  inches),  showing  a  smaller  area  for  a  compa¬ 
rable  scale  than  that  shown  on  the  Robertson  system.  The  OSL  chart  scales  were  labeled  as 
one  half  the  vertical  or  horizontal  dimension  of  the  chart,  a  specification  analogous  to  the 
radius  of  a  radar  image,  as  1  nautical  mile  (nm),  etc.  The  available  scales  are  listed  in  the 
fourth  column  of  Figure  6-1.  The  fifth  column  shows  the  geographic  areas  that  would  be 
visible  on  the  main  chart  of  the  OSL  at  scales  that  would  correspond  to  those  most  fre¬ 
quently  chosen  on  the  Robertson. 


The  most  frequently  chosen  OSL  scales,  in  the  sixth  column  of  the  figure  tend  to  cluster 
around  those  most  frequently  chosen  on  the  Robertson.  This  tendency  suggests  that  the 
mariners  had  a  preference  for  familiar  chart  scales. 

When  using  the  OSL  ECDIS,  mariners  indicated  that  having  two  chart  windows  on  the  dis¬ 
play  was  quite  useful.  Some  of  the  mariners  preferred  to  keep  the  smaller,  secondary  chart 
at  a  relatively  small  scale,  so  that  at  a  glance  they  could  see  a  larger  portion  of  the  area  cov¬ 
ered  by  the  digitized  chart  than  that  shown  on  the  main  chart.  The  mariners’  use  of  this  ar¬ 
rangement  was  chosen  with  some  frequency,  as  indicated  in  the  last  column  of  the  figiore. 
Some  of  the  mariners  found  a  different  use  for  the  second  chart:  they  used  a  very  large  scale 
(smaller  radius)  that  showed  only  the  planned  track  and  the  scaled  outline  of  own  ship.  This 
arrangement  is  a  graphic  presentation  of  the  cross-track  distance  indicated  in  the  alphanu¬ 
meric  display.  It  could  be  considered  an  approximation  to  a  steering  display,  rather  than  a 
navigation  display.  The  relatively  high  frequencies  for  the  small  radius  scales  are  the  result 
of  only  a  few  mariners  making  that  selection  a  number  of  times.  In  general,  changes  in 
scale  of  the  secondary  chart  were  made  depending  on  location  in  the  waterway. 

6.3  USE  OF  ELECTRONIC  CHART  FEATURES 

A  consideration  in  the  selection  of  the  two  ECDIS  systems  was  the  differences  in  the  pre¬ 
sentations  and  content  of  their  electronic  charts.  The  Robertson  electronic  chart  was  more 
complex  and  more  closely  imitated  a  paper  chart.  Its  permanently  retained  features 
included  the  coastline,  several  pre-defined  depth  contours,  traffic  routing  systems,  and  an 
indication  of  the  selected  scale  (the  latter  as  part  of  the  separate  LCD  panel).  In  addition,  six 
independent  layers  of  chart  data  could  be  added  including  buoy  symbols,  depth  soundings,  a 
latitude/longitude  grid  with  scale  bar,  two  levels  of  text  (major  place  names,  and  minor 
place  names  with  aids  to  navigation  characteristics),  indication  of  lighted  aids,  and 
indication  of  submerged  wrecks  and  other  hazards. 

The  permanently  retained  features  of  the  simpler  OSL  electronic  charts  included  the  coast¬ 
line,  several  pre-defined  depth  contours,  and  an  indication  of  the  selected  scale  (the  latter  as 
a  part  of  the  separate  touch  screen  panel).  Other  features  that  could  be  added  to  the  display 
included  buoy  symbols,  compass  rose,  north  arrow,  and  a  visible  bearing  line.  The  charac¬ 
teristics  of  the  two  ECDIS  systems  are  described  more  completely  in  Appendix  B. 

Data  collection  methods  used  to  examine  the  mariners’  use  of,  and  reactions  to,  the  elec¬ 
tronic  chart  presentations  included: 
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1. )  an  experimenter’s  tally  (via  remote  video  camera)  of  the  chart  features  enabled/dis¬ 

enabled 

2. )  the  post-scenario  questionnaire  in  which  the  mariner  selected  features  used  from  a 

list  of  those  available  and  added  other  features  desired 

3. )  the  paper  chart  use  form  on  which  the  mariner  recorded  any  information  used 

4. )  the  post-experiment  questionnaire  in  which  the  mariner  gave  his  opinion  on  whether 

to  display  various  chart  features  "always,"  "at  the  users  option,'  or  never 

6.3.1  F.xpftrimenter’s  Tally  of  Electronic  Chart  Features  Used 

Table  6-1  shows  the  chart  features  that  could  be  manipulated  on  each  ECDIS  system  and  the 
frequency  with  which  each  was  enabled/dis-enabled  for  the  Robertson  and  OSL  ECDIS  sce¬ 
narios.  For  the  four  scenarios  using  the  Robertson  ECDIS,  the  mariners  kept  the  beacons, 
buoys,  danger  symbols,  and  the  main  level  of  text  displayed  most  of  the  time.  Mariners 
tended  to  enable  both  the  second  level  of  text  and  the  depth  soundings  to  read  the  informa¬ 
tion  and  then  immediately  dis-enable  them.  Many  mariners  enabled  the  latitude/longitude 
grid  only  when  they  reached  the  channel.  The  relatively  high  frequencies  shown  in  the  table 
for  the  Robertson  depth  soundings,  text,  and  grid  tallies  were  accumulated  as  a  result  of  one 
mariner's  attempt  to  orient  himself  during  the  manual  update  scenario,  contributing  23, 12, 
and  12  tally  points  to  those  three  features,  respectively.  Otherwise,  each  feature  was  turned 
on  or  off  only  one  to  five  times  per  scenario. 

For  the  three  scenarios  using  the  OSL  ECDIS,  mariners  always  kept  the  buoy  symbols  en¬ 
abled,  although  this  might  have  been  because  a  rather  complicated  procedure  was  involved 
to  enable/dis-enable  them.  The  compass  rose  and  north  arrow  were  generally  kept  enabled, 
though  the  mariners  tended  to  enable/dis-enable  the  visible  bearing  line  between  one  and 
five  times  per  scenario. 

6.3.2  Post-Scenario  Questionnaire  on  Electronic  Chart  Features  Used 

The  post-scenario  questionnaire  presented  a  list  of  features  and  asked  the  mariner  to  indicate 
which  features  he  had  actually  used.  For  the  most  part,  the  mariners  all  indicated  that  they 
used  the  traffic  lanes,  buoy  symbols,  light  characteristics  and  depth  markings  during  each  of 
the  four  Robertson  scenarios.  In  only  about  one-third  of  the  transits  did  the  mariners 
indicate  that  they  used  the  danger  markings,  geographic  names,  grid,  and  aids  to  navigation 
characteristics. 
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Table  6- 1 .  Observed  U  se  of  Electronic  Chart  Features 


Robertson  Chart 
Feature 

#  times 
turned 
on/off 

OSL  Chart 
Feature 

#  times 
turned 
on/off 

Beacons 
Buoy  Symbols 
Danger  Symbols(Marks) 
Depth  Soundings 
Text 
Grid 

37 

Compass  rose 
North  arrow 
Visible  bearing  line 
Change  chart  layout 

4 

41 

4 

46 

11 

71 

3 

67 

45 

The  additional  chart  features  mariners  most  commonly  indicated  that  they  would  have 
wanted  on  the  ECDIS  include  buoy  names  and  numbers,  information  on  bridges  and  an¬ 
chorage  locations,  numerical  indication  of  the  depths  associated  with  the  contours,  and  the 
names  of  strategic  geographic  points.  See  Table  E-3  in  Appendix  E  for  a  complete  list  of 
the  features  mentioned  by  each  mariner.  Note  that  Table  E-3  also  itemizes  the  information 
each  mariner  took  from  the  paper  chart  discussed  in  Section  6.3.3  below  as  well  as  addi¬ 
tional  navigational  data  required  as  discussed  in  Section  6.4.2. 

6.3.3  Paper  Chart  Information  Used 

During  the  ECDIS  scenarios,  the  relevant  paper  charts  (on  which  a  passage  plan  had  been 
prepared  by  the  experimental  staff)  were  available  if  necessary.  The  mariners  were  asked  to 
record  any  information  they  took  from  the  paper  chart  during  the  scenario  on  a  "paper  chart 
use"  form. 

By  far,  the  most  predominant  use  of  the  paper  chart  was  to  reference  the  buoy  numbers  in 
Ambrose  and  San  Francisco  channels.  Buoy  names  and  points  of  land  used  for  taking  fixes 
were  also  referenced  fairly  often,  as  were  depth  soundings  (for  comparison  to  Robertson 
soundings)  and  occasional  position  plotting  to  confirm  the  vessel's  location.  Indeed,  several 
mariners  mentioned  that  buoy  names  and  numbers  must  be  accessible  for  reference  during 
ship-to-ship  and  ship-to-shore  communications.  Mariners  also  mentioned  that  while  some 
text  was  available  on  the  Robertson  ECDIS,  more  consideration  must  be  given  to  choosing, 
from  the  mariner's  perspective,  the  most  useful  points  of  land,  etc.  to  label. 
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6.3.4  Pnst-F.xneriment  Questionnaire  on  Disnlav  of  Chart  Features 


The  mariners  were  asked  to  give  their  recommendation  on  when  the  EQ)IS  should  display 
each  of  the  chart  features  listed  in  Table  6-2.  Table  6-2  shows  the  number  of  mariners  (out 
of  six)  who  indicated  that  each  feature:  should  be  displayed  always,  should  be  turned  on  or 
off  with  a  single  keystroke  at  the  users  option,  or  should  never  be  displayed.  The  table  has 
been  re-organized  here  to  highlight  the  areas  of  consensus  among  the  mariners  who  partici¬ 
pated  in  the  study. 

The  mariners  were  also  asked  to  choose  the  three  most  important  and  three  least  important 
features  for  an  electronic  chart,  from  the  list  given  in  Table  6-2.  Table  6-3  presents  those 
features  chosen  by  the  mariners.  Note  that  there  is  some  consensus  about  which  are  most 
important. 


6.4  ECDIS-BASED  NAVIGATION  FUNCTIONS 

The  Robertson  ECDIS  provided  a  variety  of  navigational  data  and  functions  including 
course,  speed,  course  and  speed  made  good,  set,  drift,  vessel  and  cursor  lat/long,  and 
range/bearing  to  the  cursor.  Data  available  on  selected  ARPA  targets  (one  at  a  time)  in¬ 
cluded  range/bearing,  CPA  and  TCPA.  When  the  "automated  navigation  tracking  system" 
was  enabled,  data  were  available  including  cross-track  error,  course  to  steer,  distance  to  go, 
rate  of  turn,  and  ETA,  along  with  a  variety  of  warning  alarms.  Other  navigational  data  were 
also  available  including  true  or  relative  heading/speed  vectors  for  targets  and  own  ship,  dead 
reckoning  position  reset,  track  history  with  or  without  time  labels,  and  a  mechanism  for  nu¬ 
merically  inputting  own  ship’s  course,  speed,  set  and  drift. 

The  OSL  ECDIS  also  provided  a  variety  of  navigational  data  and  functions  including  vessel 
and  cursor  lat/long,  gyro,  range/bearing  to  the  cursor  and  to  up  to  four  targets,  cross-track 
error,  course  to  steer,  a  position  offset  function  for  manual  positioning,  and  a  full  radar 
overlay  with  gain  control. 
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Table  6-2.  Mariners’  Recommendations  of  Charted  Features 


Display 

At  Users 

Display 

Always 

Option 

Never 

CHARTED  FEATURES 

(#  of  mariners) 

coastline/landmass 

6 

0 

0 

indication  of  fixed  aids  to  navigation 

6 

0 

0 

indication  of  floating  aids  to  navigation 

6 

0 

0 

Federal  channel  lines 

4 

2 

0 

navigation  lanes/fairways 

4 

2 

0 

pilot  areas 

4 

2 

0 

indication  of  isolat^  dangers 

4 

2 

0 

spot  soundings 

0 

6 

0 

names-landmasses,  islands,  points, etc. 

1 

5 

0 

light  /  sound  characteristics 

1 

5 

0 

cable^ipeline  areas 

1 

5 

0 

details  of  isolated  dangers 

1 

5 

0 

Lat/Long  grid  lines 

1 

5 

0 

bottom  characteristics 

0 

5 

1 

details  of  cautionary  notes 

0 

5 

1 

ENC  edition  date 

0 

5 

0 

anchorages 

2 

4 

0 

bottom  contours 

2 

4 

0 

compass  rose 

2 

4 

0 

physical  classification  (can/nun) 

1 

4 

1 

physical  description  (e.g.  white  tower) 

1 

4 

1 

magnetic  variation 

0 

4 

2 

geodetic  datum 

4 

2 

prohibited  and  restricted  areas 

3 

3 

0 

indication  of  cautionary  notes 

3 

3 

0 

indication  of  units  of  depths  and  heights 

3 

3 

0 

radio  characteristics  (RACON) 

2 

3 

1 

coastal  topography 

1 

3 

2 

land  feature/characteristics 

1 

3 

2 

visual  and  radar  conspicuous  features 

2 

2 

2 

Three  methods  were  used  to  collect  data  for  an  analysis  of  ECDIS-based  navigation  func¬ 
tions.  These  included: 

1. )  an  experimenter's  tally  of  the  functions  used  by  the  mariner  during  each  scenario 

2. )  post-scenario  confirmation  by  the  mariner  of  the  functions  used  and  additional 

functions  required 

3. )  a  post-experiment  questionnaire  in  which  the  mariner  gave  his  opinion  on  whether  to 

display  various  ECDIS -generated  information  "always,"  "at  the  users  option,"  or 
"never” 
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Table  6-3.  Mariners’  Selection  of  Most  and  Least  Important  Charted  Features 


Chart  Features  MOST  Important 

#of 

mariners 

n 

Chart  Features  LEAST  Important 

#of 

mariners 

indication  of  floating  aids  to  navigation 
indication  of  fixed  aids  to  navigation 
navigation  lanes/fairways 
coastlin^andmass 
spot  soundings 
coastal  topography 
bottom  contours 

5 

geodetic  datum 
bottom  characteristics 
coastal  topography 
magnetic  variation 
anchorages 
details  of  cautionary  notes 
ENC  edition  date 
physical  classification  (can/nun) 
physical  description  (e.g.  white  tower) 

pilot  areas 

4 

4 

3 

4 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

6.4.1  Experimenter's  Tally  of  ECDIS-Based  Navigation  Functions  Used 

Most  of  the  navigational  data  mentioned  above  were  continuously  visible  on  the  text  display 
of  each  ECDIS,  making  it  impossible  for  an  experimenter  to  determine  whether  or  not  the 
data  were  being  used.  Discussion  of  navigational  data  used,  therefore,  is  presented  below  in 
the  summary  of  the  post-scenario  questionnaires.  The  other  navigational  functions  were 
generally  accessed  through  a  series  of  sub-menus  on  each  ECDIS.  Table  6-4  presents  the 
frequency  with  which  each  of  the  navigational  functions  was  accessed,  as  observed  via  the 
remote  video  camera  system. 

With  the  OSL  ECDIS,  relatively  few  options  were  available  to  the  mariner  in  terms  of  navi¬ 
gational  functions.  For  the  most  part,  the  mariner  tended  to  simply  monitor  the  display, 
without  making  a  lot  of  adjustments.  Mariners  did  use  the  cursor  quite  often  to  get  ranges 
and  bearings,  but  since  there  was  not  a  visibly  identifiable  mode  for  this  function,  occur¬ 
rences  were  not  recorded  by  the  experimenter. 

The  mariners  tended  to  simply  monitor  the  Robertson  ECDIS  as  well,  though  more  func¬ 
tions  could  be  and  were  accessed  as  can  be  seen  in  Table  6-4. 

6.4.2  Post-Scenario  Questionnaire  on  ECDIS-Based  Navigation  Functions  Used 

The  experimenter’s  tally  was  supplemented  by  asking  the  mariner  to  check  off  from  a  list  in 
the  post-scenario  questionnaire,  the  types  of  navigational  data  actually  used  during  each 
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Table  6-4.  Observed  Use  of  ECDIS-Based  Navigation  Functions 


OSL  Navigational 
Functions 

frequency 

Robertson  Navigational 
Functions 

frequency 

Enable  ARPA  targets 

6 

target  vectors  on/off 

3 

drop  radar  targets 

9 

target  vectors  true/relative 

16 

adjust  radar  gain 

10 

Enable  ANTS 

37 

DR  Reset  (using  cursor) 

114 

ETA 

99 

set  position  error  tolerance 

1 

set  course  to  steer 
error  tolerance 

1 

set  cross-track  error  toler¬ 
ance 

4 

nav  lines 

87 

get  ARPA  target  information 

35 

range/bearing 

170 

manually  center  chart 

52 

own  ship’s  vector  on/off 

10 

own  ship’s  vector  (in  min¬ 
utes) 

24 

Track  history  on/off 

17 

track  history  time  labels 
on/off 

12 

DR  reset  (using  cursor) 

64 

numerically  input 
own  ship’s  drift 

6 

numerically  input 
own  ship's  set 

6 

i 

numerically  input 
own  ship’s  DR  position 

3 

scenario.  For  the  three  QSL  ECDIS  scenarios,  the  mariners  indicated  that  they  all  used  both 
the  range/bearing  to  the  cursor  function  and  the  course-to-steer  data.  They  used  the  radar 
overlay  and  target  information  in  both  of  the  scenarios  in  which  it  was  available,  and  the 
gyro  and  cross-track  distance  data  in  50  to  75  percent  of  the  scenarios.  The  mariners  also 
indicated  that  they  never  used  the  latitude/longitude  of  the  cursor's  location  or  of  the  vessel's 
location. 

Across  the  four  Robertson  ECDIS  scenarios,  the  mariners  indicated  that  they  all  used  the  in¬ 
formation  on  course,  speed,  ETA  to  next  waypoint,  ship's  vector,  and  distance-to-go,  as  well 
as  the  range/bearing  to  the  cursor  function.  In  the  scenario  with  integrated  ARPA  targets 
available,  data  on  selected  targets  was  used  by  all  of  the  mariners.  The  track  time/history 
was  used  in  about  half  of  the  scenarios,  as  were  the  course  and  speed  made  good  data. 
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Apparently,  the  course-  and  speed-made-good  information  as  well  as  set  and  drift  data 
worked  only  intermittently  during  the  experiment,  as  the  mariners  often  indicated  that  they 
would  have  used  these  data  if  available. 

The  most  commonly  mentioned  additional  data  and  navigational  functions  the  mariners 
wanted  include  ETA  to  next  waypoint,  ARPA  target  vectors  (mentioned  after  OSL  scenar¬ 
ios),  course-  and  speed-made-good,  course  change  and  chart  change  alarms,  set/drift  with 
mechanism  to  adjust  during  DR  mode,  and  CPA/TCPA  to  the  cursor.  A  complete  list  of  the 
items  mentioned  can  be  found  in  Table  E-3  in  Appendix  E. 

6.4.3  Pnst-Exneriment  Questionnaire  on  ECDIS-Based  Navigation  Functions 

As  was  the  case  for  the  data  on  electronic  chart  features  presented  above,  the  mariners  were 
asked  to  give  their  expert  opinion  on  when  the  ECDIS  should  display  each  type  of  ECDIS- 
generated  information  listed  in  Table  6-5.  Table  6-5  below  shows  the  number  of  mariners 
(out  of  six)  who  indicated  whether  each  item:  should  be  generated  and  displayed  always, 
should  be  turned  on  or  off  with  a  single  keystroke  at  the  users  option,  or  should  never  be 
displayed.  The  table  has  been  re-organized  here  to  highlight  the  areas  of  consensus  among 
the  mariners  who  participated  in  the  study. 

The  mariners  were  also  asked  to  choose  the  three  most  important  and  three  least  important 
types  of  EODIS-generated  information,  from  the  list  given  in  Table  6-5.  Table  6-6  presents 
those  features  chosen  by  the  mariners. 
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Table  6-5. 


Mariners’  Recommendations  of  ECDIS-Based  Navigation  Functions 


EQDIS-Generated  Information 


navigation  fault  alam  (e.g.  GPS  down) 
own  ship  outline 
display  planned  trackline 
display  waypoint  and  waypoint  number 

Past  track 

vector  of  course  and  speed  made  good 
display  overlay  of  actual  radar 
Set  and  Drift 
display  range  rings 
vector  of  own  ship  he^ng  and  speed 
display  selected  ARPA  targets 
display  current  vectors 
ETA  to  waypoint 
display  dead  reckoned  position  and  time 

scale  bar 
chart  scale  boundaries 
display  wheel  over  points/tum  radius 
grounding  alarm 
display  own  ship's  safety  depth  contour 
zoom  in/  out  function 
display  chart  north  up  and  course  up 
off  track  alarm 
display  fix  marker  and  time 
course  to  steer  (trackline) 
provide  method  for  manual  fix  taking 
display  visual  limits  of  lights 


Display 

Never 


(#  of  mariners) 

6 

0 

Hi 

5 

1 

5 

1 

BH 

4 

2 

9H 

1 

5 

0 

1 

5 

0 

1 

5 

0 

1 

5 

0 

0 

5 

1 

2 

4 

0 

2 

4 

0 

1 

4 

1 

1 

4 

1 

1 

4 

0 

1 

4 

0 

1 

4 

0 

3 

3 

0 

3 

2 

1 

3 

1 

1 

3 

1 

1 

2 

3 

0 

2 

3 

1 

1 

3 

0 

1 

3 

2 

1 

3 

2 

Table  6-6.  Mariners’  Selection  of  Most  and  Least  Important  ECDIS  Generated 
Information 


MOST  Important 
ECDIS-Generated  Information 

#of 

mariners 

LEAST  Important 
ECDIS-Generated  Information 

#of 

mariners 

display  planned  trackline 
nav  fault  alarm  (e.g.  GPS  down) 
display  selected  ARPA  targets 
vector  of  own  ship  heading  /  speed 
course  to  steer  (trackline) 
own  ship  outline 
display  wheel  over  points/tum  radius 

Past  track 

vector  of  course  and  speed  made  good 

4 

display  visual  limits  of  lights 
course  to  steer  (trackline) 
display  range  rings 
zoom  in/  out  function 
provide  for  manual  fix  taking 
display  current  vectors 
own  ship's  safety  depth  contour 
display  wheel  over  points/tum  radius 
off  track  alarm 
Past  track 

vector  of  own  ship  heading  and  speed 

3 

3 

2 

2 

2 

2 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 
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7.0  RESULTS  AND  DISCUSSION:  ROLE  OF  ECDIS-BASED  RADAR 
FEATURES  IN  ROUTE  MONITORING 

7. 1  RADAR,  ECDIS,  AND  ECDIS-BASED  RADAR  FEATURES  IN  REDUCED 
VISIBILITY 

7.1.1  Thft  Characteristics  of  the  ECDIS-Based  Radar  Features 

One  of  the  objectives  of  the  experiment  was  to  examine  the  contribution  that  the  integra¬ 
tion  of  radar  features  on  an  ECDIS  device  might  make  during  route  monitoring.  Radar 
features  were  among  the  considerations  in  selecting  the  experimental  ECDIS  devices.  To 
review,  the  Offshore  Systems  Limited  (OSL)  device  integrated  a  complete  radar  video 
with  the  relatively  simple  chart  image  in  its  graphic  window  and  presented  partial  target 
information  (range  and  bearing  of  up  to  four  targets)  in  the  alphanumeric  window.  The 
Robertson  Disc  Navigation  device  was  integrated  with  the  ARPA  unit  and  received  data 
from  the  ARPA  about  acquired  targets.  These  targets  were  displayed  on  the  Robertson’s 
more  chart-like  screen,  while  the  range,  bearing,  CPA,  and  TCPA  of  a  selected  target  was 
displayed  on  a  separate  alphanumeric  screen.  In  addition,  the  Robertson  device  allowed 
the  user  to  add  true  or  relative  heading  vectors  to  the  targets.  These  devices  are  described 
more  completely  in  Section  2.2  and  in  Appendix  B. 

7.1.2  Mariner  Preference  for  ECDIS  with  Radar  Features  for  Navigation 

Each  radar  was  run  for  one  scenario  (Scenarios  1  and  4)  in  reduced  visibility  (one 
nautical  mile)  to  allow  the  participating  mariners  to  examine  and  experience  these 
different  presentations.  The  mariners’  reports  of  the  primary  method  of  navigation  used 
in  each  segment  of  those  transits  is  summarized  in  Table  7-1.  These  data  are  a  further 
breakdown  of  the  data  presented  in  Table  5-1.  Reports  on  the  Robertson  device  run  in 
reduced  visibility  without  such  features  (Scenario  7)  are  also  included  to  provide  a 
baseline  of  preference  for  ECDIS  without  radar  integration.  There  does  appear  to  be  a 
greater  preference  for  ECDIS  with  radar  integration,  especially  in  the  coastal  phase  of 
navigation.  The  major  competitor  for  the  use  of  ECDIS  for  navigation  in  the  coastal 
phase  of  the  transit  is  the  radar/ARPA  on  the  bridge.  In  the  harbor,  both  the  radar/ARPA 
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Table  1-1.  Primary  Method  of  Navigation  in  Reduced  Visibility 
as  Reported  by  Mariners 


PROPORTION  OF  SEGMENTS  USING  METHOD 
IN  EACH  BRIDGE  CONDITION 

BRIDGE 

Conditions 

PLOTTING/ 

PAPER 

CHART 

RADAR/ 

ARPA 

VISUAL 

PILOTING 

ECDIS 

Total 

NUMBER 

SEGMENTS 

COASTAL  PHASE  OF  NAVIGATION 

OSL  with 
radar  overlay 
(Scenario  1) 

0.00 

0.00 

0.00 

1.00 

2 

Robertson  with 
ARPA  targets 
(Scenario  4) 

0.00 

0.00 

0.00 

1.00 

5 

Robertson  no 
ARPA  targets 
(Scenario  7) 

0.00 

0.75 

0.00 

0.25 

4 

HARBORMARBOR  approach  phase  of  NAVIGATION 

OSL  with 
radar  overlay 
(Scenario  1) 

0.00 

0.00 

0.12 

0.88 

8 

Robertson  with 
ARPA  targets 
(Scenario  4) 

0.00 

0.20 

0.00 

0.80 

5 

Robertson  no 
ARPA  targets 
(Scenario  7) 

0.00 

0.24 

0.08 

0.68 

25 

and  visual  piloting  are  mentioned.  The  paper  chart  is  not  mentioned.  Unfortunately,  the 
total  number  of  segments  is  very  small  which  reduces  the  reliability  of  these 
observations.  Counts  are  small  for  the  OSL  because  only  two  mariners  ran  that 
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condition;  counts  are  small  for  the  Robertson  because  that  scenario  was  a  short  transit 
with  fewer  segments. 

7.1.3  F.fFccts  on  Workload  and  Safety 

Analysis  of  the  quantitative  performance  data  collected  showed  no  differences  in  naviga¬ 
tion  workload,  collision  avoidance  workload,  or  accuracy  of  ship  tracks  that  could  be  at¬ 
tributed  to  the  presence  or  absence  of  radar  features  on  the  ECDIS.  Even  if  the  mariners 
did  have  a  preference  for  the  presence  of  the  features,  any  contributions  were  too  subtle  to 
affect  the  measmes  associated  with  navigation.  There  seems  to  be  no  obvious  reason 
why  radar  features  should  affect  navigation  workload  or  cross-track  distance  beyond  the 
effects  that  were  observed  with  the  basic  ECDIS  (as  described  in  Sections  5.2  and  5.3). 

Because  the  experimental  transits  were  designed  with  considerable  traffic,  the  mariners 
had  a  major  need  for  ARPA  for  collision  avoidance.  It  became  clear  during  the  scenarios 
that,  whether  or  not  radar  features  were  available  on  the  ECDIS,  each  mariner  made  con¬ 
tinuous  use  of  the  ARPA  unit  and  its  capabilities.  Indeed,  the  mariners  indicated  that  this 
was  precisely  because  they  required  more  ARPA-type  capabilities  than  were  available  on 
either  system.  They  did  not  consider  the  partial  ARPA  presentations  adequate  for  their 
need  and  continued  to  use  both  the  ECDIS  and  the  radar/ARPA,  with  the  result  that  there 
was  no  reduction  in  the  collision  avoidance  workload.  The  available  performance  data 
are  summarized  in  Appendix  E. 

7.2.  MARINERS”  REACTIONS  TO  RADAR  FEATURES 

The  mariners'  opinions  on  the  potential  contribution  of  integrated  radar  features  with 
ECDIS  were  expressed  during  both  the  post-scenario  debriefings  and  in  the  post-experi¬ 
ment  questionnaire.  (Samples  of  these  questionnaires  appear  in  Appendix  D.)  All  the 
mariners  felt  that  radar  features  should  be  integrated  with  ECDIS,  but  there  were  some 
differences  of  opinion  on  the  nature  of  that  integration.  Two  of  the  mariners  indicated 
that  a  user-selectable  level  of  radar  information  is  preferable:  from  none,  to  only  ARPA 
targets,  to  the  complete  radar  overlay.  One  mariner  preferred  ARPA  targets  only,  while 
another  preferred  the  radar  overlay.  Another  mariner  indicated  that  ECDIS  features  and 
functions  should  be  integrated  with  the  ARPA  rather  than  the  reverse.  His  argument  was 
that  the  primary  concern  for  mariners  is  collision  avoidance,  that  many  ECDIS  functions 
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can  akeady  be  performed  on  the  ARPA,  and  that  the  ECDIS  would  be  used  only  as  a 
monitor  anyway,  with  "the  mariner  keeping  his  head  mainly  in  the  radar." 

Both  advantages  and  disadvantages  of  having  an  integrated  radar  as  part  of  the  ECDIS 
were  mentioned.  The  most  basic  advantage  mentioned  was  that  the  mariner  would  need 
to  monitor  only  one  screen  to  obtain  all  the  information  he  needed,  and  would  not  have  to 
switch  back  and  forth  between  two  systems  (although  with  the  presentations  that  they 
saw,  the  mariners  did  switch  back  and  forth).  In  addition,  the  visual  confirmation  of  elec¬ 
tronic  chart  data  in  relation  to  the  radar  return  of  these  charted  features  would  provide 
confidence  in  the  accuracy  and  quality  of  the  ECDIS  data.  Overall,  the  mariners  indi¬ 
cated  that  integrated  chart  and  radar  would  have  a  positive  influence  on  navigational 
safety,  both  by  increasing  the  situational  awareness  of  the  conning  officer  and  by  facilitat¬ 
ing  the  identification  of  radar  targets  as  either  vessels  or  charted  objects.  The  "bird's  eye 
view"  of  the  waterway  with  heading  vectors  on  the  traffic  would  provide  the  mariner  with 
a  clearer  overall  picture  of  the  situation.  One  mariner  mentioned  that  the  strain  and  fa¬ 
tigue  associated  with  arrivals  and  departures  in  reduced  visibility  would  be  reduced,  and 
another  conjectured  that  local  authorities  might  eventually  allow  ships  so-equipped  to 
proceed  into  and  out  of  port  under  zero  visibility  conditions.  (There  were  only  minor  ef¬ 
fects  on  "situational  awareness,"  as  it  was  measured  in  this  experiment.  See  Section  4.0 
and  Appendix  D.) 

A  number  of  limitations  were  also  discussed.  The  mariners  generally  agreed  that  the  po¬ 
tential  for  sea/rain  clutter  to  obliterate  chart  features  is  a  problem  that  must  be  considered. 
Each  mentioned  that  the  presentation  of  the  radar  return  that  they  saw  was  poor  and  must 
be  improved,  at  least  to  the  current  level  of  technology.  Moreover,  additional  ARPA  data 
and  capabilities  were  considered  essential  in  order  for  any  degree  of  radar  integration  to 
be  useful.  The  mariners  were  emphatic  about  needing  “an  excellent,  fully  capable 
radar/ARPA  at  all  times.”  During  the  Robertson  scenarios  in  which  targets  acquired  on 
the  ARPA  were  automatically  transferred  to  the  ECDIS,  it  was  noted  that  there  were  po¬ 
tential  dangers  with  this  arrangement.  Even  in  an  "auto-acquire"  mode,  the  ARPA  does 
not  always  acquire  every  target  and  often  acquires  inappropriate  ones.  Moreover,  the 
ARPA  can  lose  targets  on  occasion.  They  indicated  that  a  mariner  could  get  into  serious 
trouble  if  he  got  absorbed  in  watching  the  ECDIS  display,  thinking  that  he  had  informa¬ 
tion  on  all  of  the  existing  traffic  and  ignoring  the  radar.  One  mariner  also  indicated  con¬ 
cern  that  radar  integration  might  cause  a  mariner,  concentrating  on  the  act  of  collision 
avoidance,  to  be  distracted  from  the  act  of  navigation. 
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The  basic  objectives  of  a  radar  overlay  integrated  with  ECDIS  need  to  be  considered 
carefully  in  order  to  maximize  its  contribution  to  the  integrated  system.  Three  potential 
objectives  are:  to  check  on  the  accuracy  of  position  information,  to  position  own  ship 
during  a  manual  position  update  procedure,  and  to  replace  radar/ARPA  during  collision 
avoidance.  Each  of  these  objectives  requires  a  different  level  of  radar  information  to  be 
integrated.  The  mariners  rated  both  the  usefulness  and  safety  of  using  radar  (or  target) 
overlay  to  accomplish  each  of  these  objectives.  These  ratings  were  done  after  experienc¬ 
ing  each  of  the  scenarios  in  which  radar  information  was  integrated  with  the  ECDIS 
(Scenarios  1, 3,  and  4).  Ratings  varied  on  a  "1”  to  "5"  scale  between  "not  use- 
ful"/"unsafe"  to  "very  useful"/"very  safe".  Table  7-2  shows  the  mean  ratings  the  mariners 
gave  for  each  of  these  three  objectives.  The  data  from  individual  scenarios  are  also  given 
since  the  number  of  responses  is  quite  variable  and  the  means  quite  different  between 
scenarios.  (In  Scenario  3  they  used  the  radar  overlay  as  a  mechanism  to  manually  update 
own  ship’s  position.)  (The  rating  scale  as  they  saw  it  appears  in  the  post-scenario  ques¬ 
tionnaire  presented  in  Appendix  D.) 

Overall,  the  mariners  felt  that  using  the  radar  overlay  for  checking  the  accuracy  of  their 
position  is  both  a  useful  and  a  safe  procedure.  In  each  of  the  scenarios,  the  mean  ratings 
were  4.0  and  4.33  or  better,  for  usefulness  and  safety,  respectively. 


For  collision  avoidance  purposes,  however,  opinion  varied  to  both  ends  of  the  scale.  The 
OSL  full  radar  overlay  (Scenario  1)  was  considered  not  useful  and  very  unsafe  as  it  was 
presented  during  the  experiment  (mean  ratings  were  1.17  and  1.00,  respectively).  The 
Robertson  target  overlay  (Scenario  4)  was  generally  considered  to  be  very  useful  for  col¬ 
lision  avoidance  (mean  rating  was  4.5),  but  averaged  only  3.5  for  safety. 

As  the  primary  method  for  manually  updating  the  vessel's  position  on  the  display,  scores 
on  both  usefulness  and  safety  ranged  from  1  to  5  across  the  scenarios.  Ratings  given  after 
Scenario  3,  in  which  manual  updating  was  required,  averaged  4.5  and  3.5  for  usefulness 
and  safety,  respectively,  although  unfortunately  these  numbers  include  responses  from 
only  two  mariners. 
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Table  7-2.  Mariners’  Ratings  on  the  Usefulness  and  Safety  of  Radar  Features  for 
Several  Possible  Objectives 


Radar 

Overlay 

Function: 


Check  on  Accuracy  of 
Position 

Primary  Method  of 
Manual  Positioning 

Collision 

Avoidance 

Usefulness 

Safety 

Usefulness 

Safety 

Usefulness 

Safety 

Scenarios  1, 3,  and  4  Combined 


mean 

4.18 

4.64 

3.7 

3.2 

3.32 

2.9 

standard 

(n=ll) 

(n  =  ll) 

(n  =  10) 

(n  =  10) 

(n=ll) 

(n=10) 

deviation 

0.87 

0.5 

1.42 

1.4 

1.85 

1.73 

Scenario  1  (OSL  with  Auto  Update  and  Full  Radar  Overlay) 


mean 

4.0 

5.0 

2.5 

2.5 

1.17 

1.0 

standard 

(n  =  3) 

(n  =  3) 

(n  =  2) 

(n  =  2) 

(n  =  3) 

(n  =  2) 

deviation 

1.0 

0.0 

0.71 

0.71 

0.29 

0.0 

Scenario  3  (OSL  with  Manual  Update  using  Full  Radar  Overlay) 


mean 

4.0  . 

5.0 

4.5 

3.5 

3.0 

3.0 

standard 

(n  =  2) 

(n  =  2) 

(n  =  2) 

(n  =  2) 

(n  =  2) 

(n  =  2) 

deviation 

1.41 

0.0 

0.71 

2.12 

2.83 

2.83 

Scenario  4  (Robertson  with  Auto  Update  and  Target  Overlay) 


mean 

4.33 

(n  =  6) 

standard 

deviation 

0.82 

3.83 

3.33 

4.5 

3.5 

(n  =  6) 

(n  =  6) 

(n  =  6) 

(n  =  6) 

1.6 

1.51 

0.84 

1.38 

These  data  and  the  comments  made  by  the  mariners  suggest,  first,  that  a  radar  overlay 
without  additional  ARPA  features  is  really  only  useful  (and  safe)  for  confirming  own 
ship's  position  and  does  not  aid  collision  avoidance  and,  second,  that  the  target  overlay  is 
considered  more  useful  and  safe  for  collision  avoidance,  probably  because  CPA  and 
TCPA  were  also  available  on  each  target. 
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8.0  CONCLUSIONS 

8. 1  ECDIS  IN  ROUTE  MONITORING 

8.1.1  F.fDTS  and  the  IMP  Performance  Standards 

This  study  was  designed  to  examine  a  number  of  broad  issues  from  the  IMO  Performance 
Standards  for  ECDIS  (International  Maritime  Organization,  1989,1994).  The  following  is 
a  brief  overview  of  the  conclusions  on  each  issue: 

•  ECDIS  has  the  potential  to  improve  the  safety  of  navigation,  primarily  by 
improving  the  precision  of  navigation. 

•  ECDIS  has  the  potential  to  reduce  the  workload  for  navigation,  primarily  by  freeing 
the  mariner  from  the  time-consuming  task  of  plotting  on  the  paper  chart 

•  AU  the  rharf  feanires  that  the  mariners  used  or  wanted  for  the  dynamic  process  of 
route  monitoring  are  required  by  the  IMO. 

•  The  radar  integration  on  the  experimental  systems  did  not  provide  the  freedom  from 
clutter  the  mariners  wanted  on  an  integrated  display  nor  the  complete  ARPA 
information  they  wanted  for  a  primary  radar. 

Each  of  these  issues  is  discussed  further  below.  A  listing  of  all  the  items  in  the  IMO 
Performance  Standards  for  ECDIS  (International  Maritime  Organization,  1993)  can  be 
found  in  Appendix  F.  Relevant  comments  based  on  the  findings  of  this  study  are  matched 
to  those  items. 

8.1.2  ECDIS  and  Bridge  Operations 

The  IMO/IHO  Harmonization  Group  on  ECDIS,  in  their  draft  resolution  (International 
Maritime  Organization,  1992),  stated  that  “as  ECDIS  comes  into  general  use  as  part  of  an 
overall  bridpe  system,  functions  that  extend  beyond  chart  equivalence  will  become  more 
clearly  defined.”  To  quote  one  of  the  participating  mariners,  The  ECDIS  units  used  in  the 
experiment  jierform  roughly  the  same  mechanical  function  as  the  human  navigator  working 
on  a  paper  chart;  they  take  position  data  from  an  outside  source  and  indicate  that  position  on 
a  graphical  display  of  a  portion  of  the  earth’s  surface.  The  units  are  more  than 
replacements  for  the  chart  —  they  are  replacements  for  human  activity.  The  experiment 
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reported  here  is  an  exploration  of  how  ECDIS  might  affect  the  “overall  bridge  system”  and 
“human  activity.” 


8.2  ECDIS  AND  SAFETY  OF  NAVIGATION 

The  study  showed  that  ECDIS ’s  potential  to  match  and  improve  upon  the  safety  of 
navigation,  compared  to  conventional  procedures.  Because  there  is  no  universally  accepted 
measure  of  “safety,”  a  variety  of  measures  were  examined.  With  ECDIS,  mariners  were 
able  to  achieve  a  greater  accuracy  of  navigation  as  measured  by  a  smaller  cross-track 
distance  of  the  ship  from  the  planned  track  line.  With  ECDIS,  there  was  evidence  that 
mariners  had  improved  geographic  “situational  awareness”  and  that  they  made  fewer 
navigation  “errors”  as  rated  by  an  experimenter.  With  ECDIS,  mariners  reported:  a  lower 
navigation  workload,  a  smaller  proportion  of  time  spent  on  navigation,  and,  as  a  result,  a 
higher  proportion  of  time  spent  on  the  higher  risk  collision  avoidance  task. 

Note  that  a  number  of  studies  have  demonstrated  that  navigational  displays  presenting 
cross-track  distance  can  support  highly  accurate  trackkeeping.  Such  findings  have  been  re¬ 
ported  by  a  number  of  simulator  studies  (Smith  and  Mandler,  1992)  and  verified  at  sea 
(Gonin  and  Crowell,  1992).  These  findings,  that  ECDIS  both  supports  more  accurate  ship 
control  and  allows  more  time  to  be  spent  on  non-navigation  tasks,  also  agree  with  simulator 
evaluations  of  automated  "one  man  bridge"  operation  (Schuffel,  Boer,  and  van  Breda, 
1989). 

8.3  ECDIS  AND  WORKLOAD  OF  NAVIGATION 

The  strongest  and  most  consistent  finding  was  that  the  availability  of  ECDIS  on  the  bridge 
substantially  reduced  the  mariner’s  workload  for  navigation.  This  reduction  was  demon¬ 
strated  by  the  time  distribution  reports,  by  the  National  Aeronautical  and  Space 
Administration’s  Task  Load  Index  (NASA-TLX)  (Hart  and  Staveland,  1988),  and  in  the 
mariners’  spontaneous  comments.  The  savings  in  workload  were  demonstrated  with 
automatic  updating  of  own  ship’s  position  and  without  a  requirement  to  maintain  an 
additional  plot  on  a  paper  chart. 
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8.4  ECDIS  SYSTEMS,  CHART  FEATURES,  AND  NAVIGATION  FUNCTIONS 
8.4.1  System  Requirements  in  Route  Monitoring 


The  following  requirements  were  identified  for  the  effective  use  of  ECDIS  for  route 
monitoring  in  the  Coastal  and  Harbor/Harbor  Approach  phase  of  navigation  (Federal 
Radionavigation  Plan,  1992):  accuracy  of  all  charted  information,  accuracy  and  automatic 
updating  of  own  ship’s  position  information,  the  accurate  scaling  of  own  ship’s  outline  in 
narrow  channels,  the  constant  availability  of  a  selected  subset  of  information,  the  rapid 
availability  of  a  larger  subset  of  information  for  reference,  and  the  provision  of  operator- 
friendly  navigational  functions. 

The  differences  between  the  two  commercial  devices  used  in  the  experiment  -  differences 
in  user  interface,  in  chart  presentation,  and  in  radar  features  --  seemed  quite  substantial 
during  the  planning  of  the  experiment.  While  individual  mariners  had  strong  preferences 
for  specific  features  provided  by  one  or  the  other  of  the  devices,  there  were  no  major  or 
consistent  differences  between  the  two  devices  on  any  of  the  performance  measures  exam¬ 
ined.  It  seems  appropriate  to  conclude  that  the  study  successfully  investigated  ECDIS 
principles  more  general  than  reactions  to  the  specific  devices. 

8.4.2  Charted  Features  Needed  in  Route  Monitoring 

The  findings  suggest  a  distinction  between  two  functions  of  a  navigational  chart:  a 
dynamic  function  for  route  monitoring,  which  needs  only  the  information  used  in  ship 
control,  and  a  static  function  as  a  geographic  information  system  (GIS),  which  provides 
much  more  extensive  information  for  reference. 

For  the  “dynamic”  process  of  route  monitoring,  the  mariners  used,  or  recommended,  only 
a  very  simple  subset  of  charted  features,  features  that  outline  the  safe  water  available  for  the 
transit.  The  charted  features  recommended  by  a  majority  of  mariners  for  continuous 
display  were: 

-  coastline/landmass 

-  fixed  and  floating  aids  to  navigation 

-  federal  channel  lines 

-  navigation  lanes  and  fairways 

-  isolated  dangers 
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These  features  are  all  in  the  “display  base”  or  “standard  display”  in  the  IMO  Performance 
Standards  (International  Maritime  Organization,  1994).  The  recommendation  of  only  a 
simple  display  for  route  monitoring  is  consistent  with  conclusions  in  a  number  of  other 
papers  (Smith,  1993;  Smith  and  Mandler,  1992;  Bianchetti,  1992;  and  Roeber,  1992). 

A  much  larger  subset  of  charted  features  was  recommended  to  be  user-selectable  or  was 
referenced  briefly  on  the  paper  chart  on  the  bridge.  Examples  of  the  most  frequent  of  these 
are: 

-  soundings,  depths,  bottom  contours,  etc. 

-  aids  to  navigation  characteristics,  names  of  points,  etc. 

8.4.3  ECDIS-Based  Navigation  Functions  Needed  in  Route  Monitoring 

Navigational  data  and  functions  recommended  for  continuous  display  by  a  majority  of  the 
mariners  included: 

-  planned  trackline 

-  navigation  fault  alarm  (e.g.,  GPS  down) 

-  selected  ARPA  target 

-  own  ship  outline 

-  vector  of  own  ship  course  and  speed 

Additional  functions  were  used  during  the  scenarios,  or  recommended  by  a  minority  of  the 
participating  mariners. 

Given  the  capabilities  of  a  computer,  the  functions  that  might  be  added  to  an  ECDIS  are 
limited  only  by  the  ingenuity  of  the  manufacturers.  There  should  be  concern  not  only  that 
the  functions  wanted  be  available  to  the  mariner,  but  also  that  a  system  not  be  overly 
complicated,  cluttered,  and  confused.  As  with  chart  features,  there  should  be  a  “base  set” 
of  navigation  functions  to  be  provided  in  every  mode  on  every  model  by  every 
manufacturer.  In  addition,  there  should  be  additional  functions  to  be  selectable  by  the  user. 
Manufacturers  could  have  the  opportunity  for  product  differentiation  and  innovations  as 
long  as  they  do  not  interfere  with  the  base  set.  Ideally,  there  should  be  sufficient 
standardization  across  systems  that  an  experienced  mariner  can  make  immediate,  effective 
use  of  a  different  system. 
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8.5  ECDIS,  RADAR  OVERLAY,  AND  INTEGRATED  NAVIGATION  SYSTEMS 


8.5.1  Roles  of  Radar  and  ECDIS 

The  results  of  the  experiment  were  equivocal  with  respect  to  the  contribution  of  radar 
integration.  The  mariners  did  report  using  ECDIS  with  radar  features  more  frequently 
for  navigation  than  ECDIS  without  these  features.  However,  the  analysis  found  no 
significant  differences  on  any  quantitative  performance  measure.  There  seems  to  be  no 
obvious  reason  why  radar  feamres  should  affect  navigation  workload  or  cross-track 
distance  beyond  the  effects  that  the  basic  ECDIS  were  observed  to  have.  Perhaps  other 
performance  measures  applied  under  other  conditions  would  have  found  objective 
evidence  of  an  advantage. 

Mariners  made  considerable  use  of  the  separate  radar/ARPA  on  the  bridge  for  collision 
avoidance,  finding  the  incomplete  ARPA  capabilities  on  the  two  ECDIS  devices 
inadequate  for  their  needs.  Presumably  for  this  reason,  the  radar  features  did  not  have 
a  measurable  effect  on  collision  avoidance  workload,  either. 

When  questioned,  all  the  mariners  felt  that  radar  overlay,  or  an  integration  of  radar  and 
ECDIS,  could  be  a  valuable  asset  on  the  bridge.  Possible  positive  contributions  mentioned 
included:  savings  in  workload  when  using  only  one  display,  confirmation  of  the  accuracy 
of  ECDIS  information,  and  increased  situational  awareness.  The  lack  of  these  positive  ef¬ 
fects  in  the  experiment  was  attributed  to  a  "cluttered"  display  and  to  incomplete  ARPA  in¬ 
formation  that  forced  them  to  continue  to  use  the  separate  ARPA. 

8.5.2  A  Fully  Integrated  Navigation  System 

Given  that  the  watchstanders  consider  the  radar/ARPA  to  be  critical  for  collision  avoidance 
during  route  monitoring,  the  designers  of  ECDIS  must  consider  how  these  two  systems 
will  be  used  together.  The  mariners’  comments  suggest  that  a  full  integration  would 
provide  increased  safety  and  confidence  and  reduced  workload.  A  major  concern  is  the 
very  cluttered  and  complex  display  that  could  potentially  result.  Engineering 
improvements,  such  as  high  resolution  screens  or  minimal  sea  return  and  clutter,  would 
allow  the  clearer  presentation  of  more  information  on  a  single  screen.  But,  the  final  design 
of  such  integrated  systems  is  a  human  factors  problem.  A  fully  capable  integration  of 
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radar/ARPA  and  ECDIS  should  be  done  with  careful  design  and  planning  and  with  the 
inclusion  of  the  user  in  the  process. 

The  relatively  small  subset  of  chart  information  actually  needed  during  route  monitoring 
suggests  that  the  complexity  conttibuted  by  the  chart  could  be  reduced.  Five  of  the  six 
mariners  suggested  a  simplification  of  the  chart  on  ECDIS  to  make  room  for  a  radar  over¬ 
lay.  Removing  soundings  or  water  depth  was  frequently  suggested  as  a  way  to  remove 
clutter.  A  contour  representing  “good  water”  for  the  draft  of  own  ship  was  suggested 
instead.  (This  last  feature  is  in  the  IMO  PS,  Appendix  2.)  Several  mariners  suggested  that 
the  extent  of  the  radar  overlay,  complete  video  or  ARPA  targets  only,  should  be  user- 
selectable.  The  sbcth  felt  that  a  simplified  chart  should  be  an  overlay  on  the  radar/ARPA, 
which  he  considered  the  “basic”  display.  All  agreed  that  all  chart  information  should  be 
easily  available  during  monitoring,  by  simple  operator  actions.  (This  last  feature  is  implied 
by  the  IMO  PS,  Item  6.1.)  All  chart  information  must  be  available  for  passage  planning, 
but  not  necessarily  on  the  same  display  used  for  route  monitoring. 

8.6  RESEARCH  METHODOLOGY 

8.6.1  Designing  Watch  standing  Scenarios 

The  technique  of  dividing  watchstanding  into  Navigation,  Collision  Avoidance,  and  Bridge 
Management  tasks  resulted  in  scenarios  with  good  “face  validity,”  had  good  mariner  accep¬ 
tance,  and  was  generally  effective  as  a  research  tool.  Because  the  opportunity  and  re¬ 
sources  were  not  available  for  extensive,  preliminary  testing,  there  were  unintended 
differences  in  collision  avoidance  or  navigation  workload  among  the  scenarios.  With 
revision,  the  existing  scenarios  provide  a  valuable  resource  for  future  use. 

8.6.2  Measuring  Situational  Awareness 

The  detailed  geographic  information  presented  by  ECDIS  suggested  a  hypothesis  of 
increased  “geographic”  situational  awareness  (Sarter  and  Woods,  1991)  with  its  use.  The 
procedures  developed  for  measuring  such  an  effect  showed  a  small  amount  of  sensitivity  to 
experimental  manipulations.  Unfortunately,  the  sensitivity  was  not  sufficient  to  allow  any 
reliable  conclusion.  However,  the  results  do  suggest  that,  with  further  development,  the 
measurement  of  situational  awareness  might  become  a  valuable  tool  in  the  analysis  of 
watchstanding  performance  and/or  navigation  displays. 
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8.6.3  Measuring  Mariner  Workload 


The  National  Aeronautic  and  Space  Administration’s  NASA  Task  Load  Index  (NASA- 
TLX)  had  good  mariner  acceptance  because  of  its  face  validity.  The  opportunity  to  report 
the  high  workload  made  the  (intentional)  artificiality  of  the  one-officer  Iridge  more 
acceptable  and  more  meaningful  to  the  mariners.  For  this  marine  operator/system 
evaluation,  it  proved  to  be  an  effective  research  tool  that  was  sensitive  to  the  experimental 
manipulations,  i.e.,  differences  among  scenarios  and  ECDIS  modes. 

8.6.4  A  rioser  Look  at  the  Radar  Overlay 

The  present  experiment  provided  the  mariners  with  the  opportunity  to  experience  and  react 
to  two  different  integrations  of  ECDIS  and  radar  features.  However,  the  experiment  was 
not  effective  in  measuring  any  possible  subtle  effects  that  the  partial  integration  of  radar 
might  have  on  navigation  performance.  An  experiment  that  would  be  more  likely  to  reveal 
these  effects  would  require  the  following:  greater  similarity  or  constancy  of  collision 
avoidance  or  bridge  management  conditions  across  scenarios,  preliminary  testing  of  the 
scenarios  for  equivalency  of  any  remaining  differences,  no  separate  ARPA  that  would 
allow  the  mariners  to  minimize  their  dependence  on  the  test  system,  and  a  more  extensive 
data  analysis  examining  CPA  and  distance  or  time  of  first  response  to  the  traffic  encounter. 

8.6.5  Validation  of  the  Findings  at  Sea 

Several  of  the  performance  measures  used,  and  the  conclusions  based  on  them,  could  and 
should  be  validated  by  sea  trials  using  ECDIS.  The  most  important  of  these  is  the 
mariner’s  report  of  the  distribution  of  attention  among  possible  methods  of  navigation:  pa¬ 
per  chart,  radar,  visual  piloting,  and  ECDIS.  The  paper  chart  and  ECDIS  are  the  same  on 
the  simulator  or  at  sea,  but  the  radar  and,  most  especially,  the  visual  scene  are  simplified  on 
the  simulator.  Validation  studies  done  for  earlier  versions  of  CAORF’s  visual  scene  found 
that  mariners  made  more  use  of  the  radar  and  less  use  of  the  visual  scene  on  the  simulator, 
compared  to  at-sea  performance  (Hammell,  1979). 

The  simulator  experience  with  measures  of  workload  and  accuracy  of  ship  tracks  using 
ECDIS  suggests  that  these  would  be  valuable  tools  for  sea  trials.  At  the  same  time,  their 
use  in  sea  trials  would  provide  validation  for  the  simulator  findings.  Note  that  increased 
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accuracy  of  ship  tracks,  using  the  OSL  Precision  Integrated  Navigation  System  (PINS) 
90{X),  has  been  demonstrated  at  sea  (Gonin  and  Crowell,  1992). 

Mariners’  preferences  for  charted  features  and  navigation  functions  could  also  be  validated 
at  sea.  It  is  probably  not  important  exactly  how  preference  is  measured.  On  a  real  ship, 
questionnaires  may  be  more  practical  than  video  monitors.  It  may  be  that  preferences  might 
change  with  exposure  to  the  devices  over  the  longer  time  periods  possible  on  a  real  ship. 

8.7  FURTHER  ECDIS  RELATED  ISSUES 

8.7.1  Evaluation  of  the  Operator  Interface 

Many  interface  and  display  issues  were  not  addressed  in  the  present  study.  These  fall  into 
two  broad  categories.  The  first  category  includes  such  navigation  display  issues  as;  screen 
resolution  needed  to  present  navigation  data,  the  speed  of  re-draw  needed  for  accurate  ship 
control,  appropriate  colors  and  symbols  to  carry  information,  etc.  Such  specialized  fea¬ 
tures  as  error  ellipses  and  steering  displays  might  be  included  in  this  first  category,  as  well. 
The  second  category  of  issues  is  generic  to  any  operator/computer  interface:  color  palette 
for  viewing  conditions,  controls,  data  input  devices,  etc.  These  two  categories  of  issues 
were  beyond  the  scope  of  the  present  study.  This  was  true  both  because  the  commercial 
devices  were  limited  in  the  number  of  variations  in  display  and  interface  that  could  be 
examined,  and  because  simulator  time  and  resources  were  better  spent  examining  the  more 
dynamic  issues  of  route  monitoring.  Such  issues  are  more  efficiently  examined  by  the 
application  of  existing  standards  for  interfaces,  in  the  laboratory,  or  at  sea. 

8.7.2  ECDIS  for  Route  Planning 

The  use  of  ECDIS  for  route  planning  was  examined  only  peripherally  in  this  study. 
Experimenters  prepared  the  planned  track  on  the  ECDIS  systems  both  to  control  the  tracks 
taken  during  the  scenarios  and  to  allow  the  test  mariners  to  spend  more  time  on  the  simula¬ 
tor.  The  use  of  ECDIS  for  route  planning  is  an  issue  that  could  be  better  examined  in  a 
setting  where  mariners  would  have  the  use  of  the  system  over  longer  periods  of  time. 

Their  experience  would  identify  the  charted  features  and  navigation  functions  needed. 
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8.7.3.  F.fFpcts  of  EmTS  on  Collision  Avoidance  and  Bridge  Management 


The  effects  of  ECDIS  on  the  safety  of  collision  avoidance  have  not  been  fully  examined. 
There  is  evidence  from  earlier  simulator  experiments  that  mariners  using  navigational 
displays  with  the  resulting  greater  knowledge  of  their  position  tend  to  make  greater 
excursions  from  the  planned  track  to  avoid  traffic  encounters  (Schryver,  1983,  Aranow, 
1979;  Hayes,  1979;  CAORF  Research  Staff,  1978b).  A  simulator  study  has  also  found 
larger  distances  from  obstructions  (e.g.,  bridge  pilings)  (O’Hara  and  Brown,  1985). 

This  study  found  no  effects  of  ECDIS  on  the  bridge  management  workload.  A  system 
based  on  a  microprocessor  could  provide  such  management  capabilities  as  recording  track 
histories,  logging  communications,  keeping  notes  accumulated  during  passage  planning  or 
earlier  passages,  etc. 

8.7.4  Training  for  ECDIS  Use 

The  issue  of  training  for  ECDIS  use  was  dealt  with  only  very  peripheraUy  in  this  study. 

The  test  mariners  were  given  brief  training  and  testing  on  the  use  of  each  of  the  systems  be¬ 
fore  the  experimental  transits.  Their  responses  on  the  questionnaire  indicated  that  this 
training  was  sufficient  for  the  purposes  of  the  experiment  and  that  they  did  not  think  that 
special  training  and  certification  would  be  needed  for  ECDIS  use.  However,  their  need  for 
an  understanding  of  the  system  was  limited  because  they  did  not  do  passage  planning  or 
concern  themselves  with  maintenance,  chart  updating,  etc.  The  issue  of  training  for  this 
new  technology  is  a  complex  one  which  is  being  dealt  with  elsewhere  (Flinn  and  Stewart, 
1993;  Sanquist,  Lee,  Mandler,  and  Rothblum,  1993;  Sanquist,  Lee,  McCallum,  and  Smith, 
1995;  and  Sanquist,  Lee,  and  Rothblum,  1994). 

8.7.5  ECDIS  as  Automation 

Many  ECDIS  issues  are  general  to  the  use  of  automated  systems:  effects  on  safety,  on 
situational  awareness,  on  workload,  on  a  need  for  special  training,  etc.  The  consequences 
of  the  increased  use  of  technology  on  ships  and  related  changes  in  the  mariner’s  role  are  of 
great  interest  in  the  marine  industry  at  the  present  time.  Many  of  the  comments  from  the 
mariners  who  participated  in  this  study  suggest  concern  that  the  consequences  might  be 
negative  as  well  as  positive:  “junior  officers”  will  be  over-confident  or  overly  complacent, 
they  will  fail  to  “keep  proper  look  out”  or  fail  to  notice  targets  not  acquired  by  ARPA,  they 
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will  not  acquire  or  will  not  maintain  the  necessary  skills  to  function  in  case  of  system 
failure,  they  will  not  be  aware  of  system  inaccuracies  or  malfunction,  “owners”  will  take  a 
man  off  the  bridge  for  every  ECDIS  they  put  on  it,  etc. 

Because  of  the  broad  implications  of  these  types  of  issues  for  maritime  safety,  the  USCG  is 
involved  in  a  major  study  of  the  effects  of  automation  (Sanquist,  Lee,  Mandler,  and 
Rothblum,  1993;  Sanquist,  Lee,  McCallum,  and  Smith,  1995;  and  Sanquist,  Lee,  and 
Rothblum,  1994).  The  study  will  develop  research  tools  to  determine  the  effects  of 
automated  systems  on  mariner  tasks  and  on  the  knowledge,  skills,  and  abilities  needed  to 
perform  those  tasks.  The  potential  use  of  simulators  in  testing  mariner  qualifications  and  in 
training  for  the  use  of  automated  systems  will  be  examined.  The  study  findings  will 
support  the  USCG  in  regulating  qualifications  for  merchant  mariners  and  in  setting  training 
requirements. 
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APPENDIX  A.O 


DESCRIPTION  OF  MSI/CAORF  FACILITY 


The  Computer  Aided  Operations  Research 
Facility  (CAORF),  located  on  die  grounds  of 
the  United  States  Merchant  Marine 
Academy,  Kings  Point,  New  York,  contains 
a  sophisticated  ship  maneuvering  simulator. 
CAORF  has  been  operated  by  MarineSafety 
International  (MSI)  since  1978  for  controlled 
research  into  man/ship/environment 
problems.  Its  main  focus  is  to  provide  a 
realistic  simulation  of  the  bridge 
environment,  ship  response,  and  waterway  to 
investigate  how  these  factors  interact  with 
and  influence  the  shiphandler's  ability  to 
maneuver  vessels  under  various  conditions. 

The  emphasis  on  the  "man-in-the-loop" 
affords  a  well  rounded  approach,  the  purpose 
of  which  is  to  examine  the  human  element  in 
marine  operations. 

Research  conducted  at  CAORF  is  sponsored 
by  either  the  U.S.  Department  of 
Transportation,  Maritime  Administration 
(MarAd)  or  clients  which  represent  industry 
or  other  government  agencies.  MarAd 
sponsored  projects  typically  address  research 
questions  relevent  to  a  wide  section  of  the 
maritime  industry.  After  a  specific  question 
is  identified,  preliminary  analysis  is  made  by 
marine  research  specialists  at  CAORF  to 
determine  whether  CAORF  on-line  or  off¬ 
line  analysis  is  required.  Based  on  these 
findings,  a  research  plan  and  detailed 
experimental  design  may  be  implemented 
and  executed,  the  results  of  which  are  freely 
publicized. 

A  similar  process  is  used  with  other  clients. 
The  client  and  CAORF  staff  will  draw  up  a 
specific  statement  of  objectives  that  defines 
the  research  plan.  Next,  a  specific  program 
may  be  implemented,  including  the 
following  tasks: 


Experimental  Design  -  definition  of  variables 
of  interest,  performance  measures,  and 
requirements  for  data  analysis. 

Planning  and  Preparation  -  development  of 
scenarios,  specifications  of  types  of  ships, 
speeds,  courses,  and  initial  positions  of  ships 
in  the  scenarios,  and  collection  of  pertinent 
data. 

Data  Base  Constniction  -  generation  of 
visual,  radar,  situation  display,  plotting,  and 
depth/cunent/bank  data  bases. 

Subject  Acquisition  -  acquisition  and 
scheduling  of  practicing  deck  officers 
(masters,  mates,  pilots)  for  participation  in 
the  scenarios  as  the  "man-in-the-loop". 


Conduct  of  Experiment  -  collection  of  data 
from  on-line  and/or  off-line  simulation. 

Data  Analysis  -  analysis  of  experimental  data 
(plots,  recorded  parameter  values,  video  tape, 
audio,  observational). 

Report  Preparation  -  presentation  of  results, 
findings/recommendations  in  final  report 
form. 

In  addition  to  the  brief  overview  of  the 
research  process  at  CAORF,  it  should  be 
noted  that  CAORF  has  the  capability  to 
simulate  any  vessel  in  any  port  or  area  in  the 
world. 

Following  is  a  detailed  description  of  the 
major  subsystems  which  comprise  CAORF, 
which  are  also  illustrated  in  Figures  A-1  and 
A-2. 
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ON-LINE  SYSTEM  HARDWARE 

Computerized  Image  Generator  -  constructs 
the  computer  generated  visual  images  of  the 
surrounding  environment. 

Images  in  full  color,  are  projected 
onto  a  cylindrical  screen  having  a 
radius  of  29  feet,  subtending  240 
degrees  horizontal  and  24  degrees 
vertical  field  of  view. 

Shading  can  be  varied,  as  can 
illumination  from  full  daylight  to 
moonless  night. 

Visibility  in  the  day  or  night 
scenarios  may  also  be  reduced  to 
simulate  any  degree  of  fog  or  haze. 

The  visual  scene  updates  30  times 
per  second  to  ensure  smooth  visual 
scene  motion. 

Perspective  is  set  for  the  actual 
bridge  height  above  the  waterline  of 
the  simulated  vessel. 

Subjective  motion  information  is 
available;  there  is  no  capacity  for 
physical  motion  simulation  at  this 
time. 

Twenty  dynamic  traffic  ships  are 
available  in  the  visual  scene. 
Numerous  stationary  ships  are  also 
available. 


Ship's  Bridge  -  a  simulated  ship's  bridge,  20 
ft.  (6.1  m)  wide  by  14  ft.  (4.3  m)  deep 
which  contains  all  equipment  and  controls 
normally  available  on  a  merchant  vessel. 

The  equipment  responds  with 
realistic  accuracy,  providing  a  subject 
watch  officer  ("man-in-the-loop")  the 
opportunity  to  maneuver  ownship 
through  a  scenario. 


Port  and  starboard  bridge  wings,  each 
equipped  with  gyro  repeaters  to  allow 
visual  bearings  to  be  taken  and 
plotted  on  a  chart. 

Tugboat  control  console  may  be 
added  for  simulation  of  a  tug 
wheelhouse. 

The  equipment  on  the  bridge  can  be 
reconfigured  for  single  or  twin  screw 
operation,  and  includes: 

Steering 

Gyro  steering,  hand  steering, 
and  NFU  steering. 

Gyro  repeater. 

Rudder  angle  indicator, 

Rate  of  turn  indicator. 
Steering  failure  alarms. 

Propulsion 

Throttle  and  telegraph 
engine  order  (single/twin 
screw), 

RPM  indicators, 

Bow/Stem  thrusters  with 
respective  indicators  and 
status  lights. 

Engine  failure  alarms. 

Ship  Motion  Indicators 

Speed  and  speed  log, 

Doppler  speed  display. 

Navigation 

2  Collision  Avoidance 
Systems,  Radar/ARPA, 
Fathometer, 

Pelorus  -  Bridge  wing 
calibrated  for  parallax. 

Wind  speed  and  direction 
indicators. 
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Communicatioiis 

VHF  radio. 

Intercom, 

Sound  powered  phones. 
Ship's  whistle. 


Sound  System 

Digitally  sampled: 

OS  engine, 

OS  whistle. 

Environmental  sounds, 
NavAids  soimds. 

Central  Data  Processor  -  computes  ship 
motion  in  accordance  with  maneuvering 
characteristics  and  environmental  conditions 
(including  currents,  channel  banks,  passing 
ship  effects). 

Models  the  behavior  of  all  other 
traffic  ships. 

Drives  the  qjpropriate  bridge 
indicators  (wind,  radar,  Doppler,  etc.) 

Communicates  with  and  controls 
visual,  radar,  sound,  and  situational 
display  subsystems. 

Drives  control  station  indicators. 

Radar  Signal  Generator  -  synthesizes  real¬ 
time  video  signals  to  stimulate  two  (2)  Plan 
Position  Indicators  (PPI's). 

Displays  up  to  20  moving  traffic 
ships. 


Control  Station  -  central  location  from  which 
the  simulator  experiment  is  initialized, 
controlled,  and  monitored. 

Traffic  ships,  assist  tugboats, 
environmental  conditions  and 
mechanical  failures  can  be  controlled 
by  operators  observing  the 
experiment  imderway. 

All  communications  between  bridge 
and  outside  persons  are  earned  out 
from  this  station. 

Assist  Tug  Simulation  -  simulation  of  up  to 
six  (6)  assist  tugs  for  use  at  any  point  along 
ownship's  hull. 

Human  Factors  Station  -  remote  location  for 
observation  of  simulator  research  in  progress. 

Unobtrusive  observation  and  data 
gathering  by  experimental 
psychologists  may  be  conducted. 

Video  and  audio  recordings  may  be 
made  of  activities  of  bridge  personnel 
for  playback/evaluation. 

OFF-LINE  SYSTEM  HARDWARE 

Fast-Time  Simulation  -  in  addition  to  real¬ 
time  simulation,  CAORF  has  the  capability 
to  perform  off-line  simulation  runs 
independent  of  visual  displays,  wheelhouse, 
radar  or  control  station  operations: 

One-ship  fast-time  routine. 

Two-ship  fast-time  routine. 


Fast-time  interactive  steering  system 
which  allows  the  user  to  control  all 
steering  from  a  CRT  monitor. 
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Figure  A- 1 .  Cutaway  of  Computer  Aided  Operations  Research  Facility  (CAORF)  Building 
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Figure  A-2.  Major  Computer  Aided  Operations  Research  Facility  (CAORF)  Subsystems 
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APPENDIX  B.O  SUPPLEMENTARY  MATERIAL  FOR 

SECTION  2.0  SIMULATION  AS  A  "TEST  BED"  FOR 
ECDIS  EVALUATION 

B.l  description  OF  TEIE  COMMERCIAL  ECDIS  DEVICES 
B.1.1  User  Interfaces  of  the  Two  Devices 

From  the  user’s  standpoint,  each  of  the  two  devices  basically  consisted  of  a  graphic 
display  and  an  operator’s  panel  for  inputting  and/or  displaying  numerical  data.  The 
Robertson  Marine  Systems’  Disc  Navigation  had  a  single  63.5  centimeter  (25  inch) 
graphic  display  of  the  chart,  while  the  Offshore  Systems  Limited’s  (QSL)  Precision 
Integrated  Navigation  System  VME  had  a  48.26  centimeter  (19  inch)  graphic  display  that 
was  portioned  into  a  main  view  and  a  smaller  secondary  view  of  the  chart,  and  a  panel  of 
numerical  navigation  data.  The  size  of  the  OSL’s  main  chart  view,  secondary  chart  view 
and  navigation  data  panel  could  be  manipulated  by  choosing  one  of  four  setup 
configurations  (described  below). 

The  operator’s  panel  on  both  devices  provided  the  user  with  some  control  over  the  data 
presented  and  the  navigational  functions  implemented.  The  operator’s  panel  on  the 
Robertson  ECDIS  was  activated  with  push-button  keys,  divided  into  different  sections. 
The  main  section  contained  an  interactive  LCD  text  display  of  navigation  data,  with  soft- 
key  push-buttons  for  accessing  several  sub-menus  of  information.  Also  part  of  this 
section  was  a  panel  of  keys  for  inputting  alphanumeric  data.  To  the  left  was  a  panel  of 
keys  for  turning  on/off  various  layers  of  chart  detail.  To  the  right  was  a  roller  ball  with 
three  associated  keys,  one  of  which  enabled  access  to  navigation  or  planning  functions 
(depending  on  mode)  requiring  cursor  positioning  with  the  roller  ball  (described  below). 
To  the  far  right  were  keys  for  controlling  system  activity  (e.g.,  on/off,  alarm 
acknowledge,  monitoring  or  planning  mode,  and  degauss). 

The  operator’s  panel  on  the  QSL  ECDIS  included  two  electro-luminescence  screens 
(ELS)  with  finger-touch  activation,  and  a  roller  ball  with  three  associated  buttons.  The 
left  touch-screen  contained  controls  for  turning  on/off  the  entire  system  and  the  radar 
overlay,  and  for  adjusting  the  radar  gain.  In  addition,  this  screen  contained  keys  for 
manipulating  the  scale  of  the  primary  and  secondary  charts.  The  right  touch-screen  was 
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divided  into  three  main  sections,  one  containing  system  information,  one  for  main  menu 
choices,  and  one  for  sub-menus.  Through  the  main  and  sub-menus,  the  navigation  data 
displayed  on  the  right  hand  portion  of  the  chart  display  screen  could  be  customized. 

The  OSL  ECDIS  provided  a  flexible  configuration  for  the  chart  display,  allowing  four 
different  layouts  on  the  screen.  They  were: 

-  "extended  nav"  consisting  of  a  large  main  chart  window  with  a  smaller  chart  window 
and  a  text  window  on  the  right  side  of  the  screen, 

-  "split  screen"  which  presented  two  graphic  windows  of  the  same  size  with  the  text 
window  across  the  bottom, 

-  "local  nav"  which  presented  one  large  chart  window  with  text  windows  on  the  right,  and 

-  "docking"  in  which  one  large  chart  display  filled  the  screen  with  no  text  windows. 

The  OSL  "extended  nav"  configuration  was  designated  as  the  default  for  the  experiment 
and  was  used  by  the  mariners  in  all  but  three  (short)  instances.  The  main  chart  window 
in  the  extended  nav  configuration  measures  26.67  cm  x  26.67  cm  (10.5  inches  x  10.5 
inches).  The  scale  of  each  of  the  two  charts  on  the  screen  could  be  manipulated 
independently  by  selecting  one  of  16  values  on  the  separate  touch  screen.  Scales  are 
specified  in  terms  of  the  geographic  distance  that  was  presented  vertically  in  the  chart 
window  (e.g.,  1/2  nm  or  25  nm).  Only  the  main  chart  window  displayed  radar  and  cursor 
data.  A  frequent  practice  was  to  use  the  main  window  for  a  large  scale  view  of  the 
immediate  area  and  the  secondary  window  for  a  small  scale,  “extended”  view.  To 
support  such  a  use,  the  secondary  OSL  chart  window  shows  an  outline  of  the  area 
covered  on  the  main  chart  window.  As  an  example,  if  the  larger  main  chart  is  scaled  to 
one  mile  and  the  smaller  secondary  chart  is  scaled  to  12  miles,  the  secondary  chart 
displays  a  square  outline  covering  a  one  mile  radius  from  the  center.  This  feature  enables 
the  mariner  to  see  at  a  glance  both  which  portion  of  the  chart  is  being  displayed  in  the 
main  window,  and  where  he  is  in  the  larger  area  of  the  entire  chart. 

B.1.2  Presentation  of  Electronic  Chart  Features  on  the  Two  Devices 

The  following  paragraphs  describe  the  OSL  and  Robertson  ECDIS  devices  as  each  was 
configured  for  the  experiment.  The  charted  features  available  on  each  system  are 
compared,  and  differences  which  are  important  to  breadth  of  coverage  of  this  experiment 
are  highlighted.  The  features  described  in  this  section  refer  to  the  monitoring  modes  of 
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each  device.  A  description  of  the  specific  features  and  functions  available  for  route 
planning  is  beyond  the  scope  of  this  report. 

Each  ECDIS  had  its  own  set  of  chart  features  that  were  not  removable,  as  well  as 
information  that  could  be  turned  on  or  off  by  the  user.  The  permanently  retained  features 
of  the  Robertson  charts  included  the  coastline,  various  pre-defined  depth  contours,  traffic 
routing  systems,  and  an  indication  of  the  selected  scale  (part  of  the  LCD  text  display).  In 
addition,  six  independent  layers  of  chart  data  could  be  added  to  the  graphic  display 
including  light  markings  to  show  which  aids  are  lighted,  danger  marks  (two  levels)  to 
show  the  location  of  wrecks  and  other  submerged  hazards,  depth  soundings,  grid,  buoy 
symbols,  and  two  levels  of  text  (major  place  names,  and  minor  place  names  with  aids  to 
navigation  characteristics).  In  addition  to  brighmess  and  contrast  controls,  the  color  of 
the  chart  features  could  be  varied  by  choosing  one  of  four  IMO-compliant  color  palettes 
to  accommodate  external  lighting  conditions. 

The  permanently  retained  features  of  the  OSL  charts  included  the  coastline,  various  pre¬ 
defined  depth  contours,  and  an  indication  of  the  selected  scale  (part  of  the  touch  screen 
panel).  Other  features  that  could  be  turned  on  or  off  through  a  sub-menu  included  buoy 
symbols,  compass  rose,  north  arrow,  and  a  visible  bearing  line  connecting  the  ship  to  the 
cursor.  (Note  that  the  presence  or  absence  of  the  visible  bearing  line  did  not  affect  the 
functioning  of  the  cursor  as  a  tool  for  obtaining  ranges  and  bearings.)  The  brighmess  and 
contrast  of  the  entire  screen  was  adjustable. 

Table  B-1  below  indicates  the  compliance  of  each  ECDIS  with  the  September  1992 
version  of  the  IMO  PPS  on  charted  features,  as  each  system  was  configured  for  the 
experiment.  In  general,  the  Robertson  electronic  chart  conformed  more  closely  to  the 
1992  Draft  Standards  in  that  it  had  the  "look"  of  a  conventional  paper  chart,  while  the 
OSL  electronic  chart  provided  less  chart  information. 
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Table  B-1 .  Compliance  of  ECDIS  Devices  with  IMO  PPS 


OSL 

ROBERTSON 

IMO  PPS  (Sept.  1992) 

PINS-VME 

DISC  NAVIGATION 

DISPLAY  BASE 

.1  coastline  (high  water) 

YES 

YES 

.2  ownship’s  safety  contour,  to  be  selected  by 

NO 

PARTIAL  The  user  could  specify 

the  mariner  from  the  depth  contours 

ship’s  draft  and  warning  time,  and 

provided  by  the  SENC 

get  auditory  warning. 

.3  indication  of  isolated  underwater  dangers  of 

NO 

NO 

depths  less  than  the  safety  contour  which  lie 
within  the  safe  waters  defined  by  the  safety 

contour 

A  indication  of  isolated  dangers  which  lie 

NO 

NO 

within  the  safe  water  defined  by  the  safety 
contour  such  as  bridges,  overhead  wires, 
etc.,  and  including  buoys  and  beacons 
whether  or  not  these  are  being  used  as  aids 
to  navigation 

.5  traffic  routing  systems 

NO 

YES  showed  traffic  separation 

scheme. 

.6  indication  of  scale,  range,  orientation  and 

PARTIAL  Scale  indicator  easily 

PARTIAL  Digital  indication  of 

display  mode 

visible  on  high-lighted  touch  panel 

scale  in  upper  left  corner  of  text 

display.  No  indication  of 

display.  No  indication  of 

orientation,  though  only  North-up 

orientation,  though  only  North-up 

was  available. 

was  available. 

.7  units  of  depth  and  height 

PARTIAL  Units  of  navig.  data  were 

NO  No  positive  indication,  though 

indicated.  There  are  depth 

depth  soundings  were  in  meters 

contours,  but  no  indication  of  what 

according  to  manuf.  Units  of 

depths  they  refer  to. 

navig.  data  were  indicated. 

STANDARD  DISPLAY 

.1  display  base 
.2  drying  line 

NO 

YES 

.3  indication  of  fixed  and  floating  aids  to 

YES  red,  white  or  green  symbols  . 

YES 

navigation 

.4  boundaries  of  fairways,  channels,  etc. 

NO 

YES 

.5  visual  and  radar  conspicuous  features 

NO 

NO 

.6  prohibited  and  restricted  areas 

NO 

NO 

.7  chart  scale  boundaries 

NO  Any  chart  could  be  scaled 

YES  Limit  defined  by  chart  type 

from  1/4  nm  to  100  nm. 

(e.g.,  coastal,  harbor). 

.8  indication  of  cautionary  notes 

NO 

NO 

SUPPLEMENTAL  INFORMATION 

.  1  spot  soundings 

NO 

YES  Can  be  turned  on/off  for 

whole  chart. 

.2  submarine  cables  and  pipelines 

NO 

NO 

.3  ferry  routes 

NO 

NO 

.4  details  of  all  isolated  dangers 

NO 

NO 

.5  details  of  aids  to  navigation 

NO 

YES  On/off  button.  Provides  nav- 

aid  characteristics  including  flash 
rate  and  color. 

.6  contents  of  cautionary  notes 

NO 

NO 

.7  ENC  edition  date 

NO 

NO 

.8  geodetic  datum 

NO 

NO 

.9  magnetic  variation 

NO 

NO 

.10  gradicule 

NO 

YES  On/off  button.  Both  grid  and 

scale  bar. 

.11  place  names 

NO 

YES  On/off  button.  Two  levels 

available:  Major  place  names  and 
then  minor  names  and  nav-aid 

characteristics  (see  .5). 
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B.1.3  Electronic  Charts  Available 


The  OSL  ECDIS  contained  three  manufacturer  digitized  charts  of  the  New  York  area 
corresponding  to  Charts  12326  (Approaches  to  New  York,  scale  1:80, 000),  12327  (New 
York  Harbor,  scale  1:40, 000)  and  12334  (lower  harbor,  scale  1:19, 000).  The  Robertson 
ECDIS  contained  two  New  York  area  charts  corresponding  to  Charts  12326  and  12327 
and  two  San  Francisco  area  charts  corresponding  to  Charts  18645  (SF  coast,  scale  1:100, 
000)  and  18649  (SF  channel  through  to  the  Southern  anchorages,  scale  1:  40,000).  The 
scales  chosen  by  the  mariners  should  be  viewed  in  the  context  of  the  areas  being  transited 
and  the  charts  that  were  available. 


B.2  SIMULATOR  AND  SIMULATION  MODEL  CONHGURATION 
B.2.1  Simulation  Geographic  Model 

Existing  simulation  models  of  New  York  (NY)  and  San  Francisco  (SF)  harbors  and 
approaches  were  used  as  operations  areas.  Both  areas  provide  high  workload,  high  risk 
coastal  and  harbor/harbor  approach  regions  of  sufficient  complexity  and  diversity  to 
investigate  ECDIS  use  over  a  range  of  conditions.  In  addition,  electronic  charts  for  the 
NY  area  were  available  from  both  vendors  and  electronic  charts  for  the  SF  area  were 
available  for  the  Robertson  Disc  Navigation  System. 

The  simulation  model  consisted  of  several  integrated  databases  that  worked  in 
coordination  to  provide  the  mariner  on  the  bridge  visual,  radar,  sound,  and  hydrodynamic 
environments  consistent  with  the  real  world  and  with  the  electronic  charts  of  the  area. 
The  ship's  bow,  buoys,  fixed  aids  to  navigation,  bridges,  land  masses,  and  cultural 
features  and  sounds  were  represented  to  the  extent  necessary  to  create  a  realistic, 
functional,  and  familiar  environment  for  the  mariner. 

B.2.2  Ship  Response  Model 

The  ship  response  model  is  a  mathematical  representation  of  a  specific  ship.  It 
reproduces  the  ship's  maneuvering  characteristics,  including  such  factors  as  the 
hydrodynamic  characteristics,  aerodynamic  profile,  and  machinery  dynamics.  It  also 
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interacts  with  the  simulation  data  base’s  representation  of  currents,  banks,  bottom,  and 
wind  to  incorporate  these  effects  into  the  ship’s  response.  A  ship  response  model  of  the 
Lancer  class  containership  was  used  during  the  entire  experiment.  This  model  has  been 
validated  against  sea  trial  data  (J.J.  Henry  Co,  Inc.  and  Systems  Control  Inc.,  1979)  and 
has  been  used  in  a  niunber  of  port  design  test  programs  (CAORF  Staff,  1992)  and  in 
senior  mariner  training  programs  (Fritzke  and  CAORF  Staff,  1985). 

The  Lancer  class  containership  is  a  medium-sized  ship  with  good  handling  characteristics 
and  is  one  with  which  all  experienced  mariners  would  feel  comfortable.  It  has  overall 
length  of  217.93  meters  (715  ft),  a  beam  of  27.43  meters  (90  ft),  and  was  modeled  with  a 
9.14  meter  (30  ft)  draft  and  displacement  of  30,130  tons.  Tables  B-2  and  B-3  present 
further  data  on  the  vessel's  characteristics. 

B.2.3  Simulator  Bridge 

The  bridge  arrangement  used  for  the  duration  of  the  study  is  shown  in  Figure  B-1.  The 
bridge  was  configured  with  the  ARP  A,  engine  order  telegraph  (EOT),  ECDIS,  and 
second  radar  unit  in  a  line  at  the  center  of  the  bridge.  The  appropriate  ECDIS  system  — 
OSL  or  Robertson  —  was  installed  for  a  particular  scenario.  The  ECDIS  on  the  bridge 
was  turned  off  during  the  interspersed  baseline  (paper  chart)  scenarios.  The  radios,  intra¬ 
ship  communications,  whistle  controls,  and  general  alarm  were  along  the  forward  and 
after  bulkheads.  The  chart  table  was  in  the  rear  on  the  port  side  of  the  bridge,  facing 
forward.  A  Differential  Global  Positioning  System  (DGPS)  receiver  and  fathometer  were 
located  behind  the  chart  table  in  bulkhead  mounted  shelves.  Other  bridge  equipment 
present  were:  gyro-repeaters,  RPM  and  rudder  angle  indicators,  course  and  rudder  angle 
recorder,  and  navigation  light  panel.  Port  and  starboard  bridge  wings  had  gyro-repeaters 
and  bearing  circles.  The  steering  stand  was  on  the  ship’s  centerline  behind  the  main 
consoles.  (A  qualified  helmsman  was  present  during  the  runs.) 

This  arrangement  was  designed  for  the  needs  of  this  experiment.  The  modified  cockpit 
arrangement  and  centralized  control  were  intended  to  take  full  advantage  of  ECDIS.  The 
mariner  could  stay  at  one  location  just  forward  of  the  steering  stand,  watch  the  ECDIS  or 
the  ARP  A,  and  perform  ship-to-ship  communications  from  that  spot.  Some  features  of 
the  bridge  arrangement  were  needed  to  facilitate  data  collection.  The  separation  of  the 
ECDIS  and  the  ARPA  required  the  mariner  to  walk  between  them,  making  it  possible  for 
experimental  observers  to  determine  which  system  the  mariner  was  using  at  any  one  time. 
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The  ECDIS  location  on  the  starboard  side  of  the  bridge  allowed  the  most  effective 
observation  by  a  ceiling  mounted  camera.  The  chart  table’s  placement  forward  of  the  aft 
bulkhead  both  allowed  the  mariner  to  look  forward  while  referring  the  paper  chart  and 
made  it  evident  to  observers  when  the  mariner  left  the  “conning”  station  to  refer  to  the 
paper  chart. 

B.2.4  Monitoring  Capabilities 

The  mariners'  activities  on  the  bridge  and  important  aspects  of  the  simulation  were 
observed  by  the  experimenters  from  the  Human  Factors  Station  (HFS).  The  entire  bridge 
area  was  monitored  via  a  network  of  six  low  light  video  cameras  and  four  microphones. 
Figure  B- 1  also  shows  the  camera  and  microphone  locations.  The  radio  and  intra-ship 
commxmications  were  also  monitored.  The  entire  progress  of  the  simulation  was 
displayed  on  video  monitors  at  the  Human  Factors  Station  (HFS).  The  simulator  s  Visual 
Situation  Display  (VSD)  provided  a  aerial  view  of  the  progress  of  own  ship  and  traffic 
ships  in  real-time  with  aids  to  navigation,  traffic  lanes,  tracklines,  and  landmasses 
displayed.  Hard  copy  printouts  of  the  VSD  display  were  systematically  collected  for  use 
in  scoring  the  mariners'  track  recreation  during  the  debriefings.  A  repeater  showing  all 
activities  on  the  ARPA  and  a  monitor  showing  own  ship  status  (i.e.,  speed,  heading, 
rudder  order/angle,  engine  revolution  per  minute,  wind,  underkeel  depth,  and  bow 
thruster  status)  were  also  located  at  the  HFS. 
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Table  B-2.  Lancer  Class  Vessel  Characteristics 


MA^  ECDIS 

Principle  Dimensions 

LOA 

218  m  (715  ft) 

LBP 

204.3  m  (670  ft) 

Beam 

27.4  m  (90  ft) 

Distance  Bridge  to  Bow 

164.6  m  (540  ft) 

Distance  Bridge  to  Stem 

53.4  m  (175  ft) 

Height  of  Eye 

19.8  m  (65  ft) 

Mean  Draft 

9.1m  (30  ft) 

Trim 

0  (even  keel) 

Displacement  at  30  feet 

30,130LTons 

Table  B-3.  EOT  -  RPM  Speed  Table 


MA^  ECDIS 


ENGINE  ORDER 

RPM 

■■■■■Rfa  at  afiH 

Full  Sea  Speed 

72 

18.5  knots 

Full  Ahead 

48 

12.5 

Half  Ahead 

30 

7.8 

Slow  Ahead 

20 

5.2 

Dead  Slow  Ahead 

10 

2.6 

Dead  Slow  Astern 

10 

Slow  Astern 

20 

Half  Astern 

30 

Full  Astern 


40 


Figure  B- 1 .  Arrangement  of  the  MSI/CAORF  Bridge  for  the  ECDIS  Evaluation 


APPENDIX  C.O  SUPPLEMENTARY  MATERIAL  FOR 

SECTION  3.0  THE  EVALUATION  PLAN 

C.  1  POSSIBLE  SEQUENCE  AND  RUN  ORDER  EFFECTS 

The  experiment  was  planned  for  each  participating  mariner  to  experience  all  nine  scenarios. 
In  such  a  case,  the  possibility  of  sequence  or  run  order  effects  is  a  consideration.  Sequence 
effects  that  might  be  expected  to  improve  performance  during  a  week  of  repeated  runs 
include  learning,  practice,  or  familiarization  with  the  simulator,  with  each  of  the  ECDIS 
systems,  and  with  the  required  bridge  management  tasks.  Effects  that  might  have  negative 
effects  on  performance  include  boredom  and  fatigue.  To  prevent  such  biases  in  measured 
performance,  each  mariner  experienced  the  scenarios  in  a  different,  pre-planned  order. 


Because  of  the  experience  and  comments  of  mariners  who  assisted  in  the  preparation  of  the 
experiment,  some  constraints  were  placed  on  a  strict  counter-balancing  of  order.  First,  to 
allow  each  mariner  to  learn  to  operate  the  ECDIS  devices,  each  mariner  ran  all  the  scenarios 
with  one  device  before  switching  to  the  other.  (There  was  training  on  each  device  just 
before  the  first  transit  with  that  device  and  the  first  device  to  be  used  was  alternated  among 
the  mariners.)  Second,  because  of  the  difficulty  in  manually  updating  ECDIS  when  its 
automatic  positioning  capability  “failed,”  this  mode  was  presented  only  after  a  mariner  had 
completed  at  least  two  ECDIS  runs  with  a  given  device.  Third,  because  of  the  possibility 
that  the  first  scenario  run  by  a  mariner  might  suffer  from  a  lack  of  familiarity  with  the 
simulator  or  with  procedures,  each  scenario  appeared  in  the  first  position  for  only  one 
mariner.  (This  first  scenario  was  after  a  “familiarization”  scenario  during  which  no  data 
were  taken.)  Lastly,  the  baseline  transits  were  spaced  so  that  they  were  in  the  different 
harbor  from  the  ECDIS  transits  before  and  after  them  to  avoid  mariner  boredom. 


These  attempts  to  avoid  bias  were  successful.  On  preliminary  analysis,  none  of  the 
quantitative  performance  measures  used  showed  any  correlation  with  run  order.  Therefore, 
order  effects  were  not  considered  in  any  subsequent  analyses. 
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C.2  THE  PARTICIPANTS:  MASTERS  AND  MATES 


Each  participant  in  the  study  was  sent  a  package  of  introductory  materials  prior  to  arrival  at 
CAORF.  The  package  included  a  brief  description  of  the  study's  focus  and  an  orientation 
to  the  week's  schedule,  a  user’s  manual  for  each  ECDIS,  a  brief  discussion  of  Differential 
GPS  (DGPS),  and  a  Mariner  Profile  questiormaire.  The  following  paragraphs  describe  the 
experience  of  the  participants  and  is  based  on  their  responses  to  a  Mariner  Profile 
questionnaire  and  any  resumes  they  appended. 

Maritime  experience.  The  six  participants  for  the  study  included  four  Masters  (unlimited 
license),  each  with  more  than  20  years  of  experience,  one  Second  Mate  (two  years  of 
experience)  whose  highest  capacity  sailed  was  Third  Mate,  and  one  Chief  Mate  (nine  years 
of  experience)  whose  highest  capacity  sailed  was  Second  Mate.  Three  of  the  four  Masters 
have  First  Class  Pilotage  in  Prince  William  Soimd,  Puget  Smmd,  and  San  Francisco  Bar  to 
the  Golden  Gate  Bridge,  respectively.  Both  Mates  also  have  a  1600  ton  Masters 
endorsement.  Five  of  the  mariners  sailed  six  to  seven  months  in  the  preceding  year,  and 
one  sailed  three  months.  The  participants'  present  sailing  vessels  include  container  ships, 
roll-on/roll-off  freighters,  liquefied  natural  gas  (LNG)  tankers,  and  crude  carriers, 
representing  companies  including  Marine  Carriers,  EXXON,  Ahrenkiel,  APL,  and  Energy 
Transport. 

Familiarity  with  ECDIS  and  navigation/plotting  devices.  The  average  reported  level  of 
familiarity  with  ECDIS  of  the  participants  prior  to  the  study  was  1.7  on  a  scale  from  1 
(unfamiliar)  to  5  (very  familiar).  However,  some  of  the  participants  indicated  that  they  did 
have  some  experience  with  or  exposure  to  other  navigation/plotter  type  devices.  This 
included  extensive  experience  of  one  Master  with  SNA-91,  a  system  that  enables  the  user 
to  digitize  chart  information  and  present  it  on  the  radar  scope,  experience  of  another  Master 
with  Trimble  Navigraphic  Loran  and  Loran/GPS,  and  one  month  of  using  Sperry 
ExxBridge  aboard  ship  by  a  Second  Mate. 

Familiarity  with  New  York  and  San  Francisco  harbors.  With  the  exception  of  one  Master 
who  sailed  into  and  out  of  New  York  Harbor  25  times  in  the  previous  12  months,  all  of  the 
participants  sailed  into  and  out  of  NY  once  or  less  in  the  last  12  months.  In  addition,  two 
of  the  Masters  and  one  Mate  sailed  into  or  out  of  San  Francisco  Harbor  six  times  in  the  last 
12  months,  while  the  rest  of  the  participants  didn't  sail  in  SF  at  all  (including  the  Master 
with  SF  Pilotage). 
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Familiarity  with  SpfttTv  R  ASCAR  radar.  With  the  exception  of  two  Masters  who  were 
unfamiUar  with  Sperry's  RASCAR  radar  (used  on  the  CAORF  bridge),  all  of  the 
participants  indicated  that  they  were  moderately  familiar  with  the  system. 

Familiarity  with  Global  Positioning  System  (GPS')  and  differential  GPS  (DOES).  Prior  to 
the  study,  the  participants'  average  reported  level  of  familiarity,  on  a  scale  from  1 
(unfamiliar)  to  5  (very  familiar),  was  3.5  for  GPS  and  2.0  for  DGPS. 

Familiarity  with  shinhandling  simulators.  All  of  the  participants  have  had  some  prior, 
recent  exposure  to  ship  simulation  at  MSI's  Newport,  RI  facihty  and/or  MSI/CAORF. 
Some  of  the  participants  have  also  used  other  simulation  facilities  such  as  Marine  Institute 
of  Technology  and  Graduate  Studies  (MITAGS)  in  Lithicum  Heights,  MD  and  Seaman's 
Church  Institute  in  New  York,  NY. 

Familiarity  with  personnel  computers.  All  of  the  Masters  indicated  that  they  use  a  personal 
computer  almost  every  day,  while  the  two  Mates  use  a  personal  computer  about  once  a 
week. 


C. 3  THE  PARTICIPATING  MARINER'S  WEEK  AT  MSI/CAORF 

Each  mariner  spent  a  full  week  making  both  arrivals  and  departures  in  New  York  and  San 
Francisco  on  the  simulator,  under  a  variety  of  very  demanding  and  realistic  conditions, 
using  ECDIS  as  the  primary  method  of  navigation.  The  performance  measures  mentioned 
below  are  described  more  fully  in  Section  4.0.  of  the  Main  Report. 

Training  with  the  first  ECDIS  device.  At  the  beginning  of  each  week,  the  participating 
mariner  received  a  thorough  and  focused  training  session  with  the  particular  ECDIS  unit 
they  would  be  using  first  (either  the  Robertson  or  the  Offshore  Systems  rmits).  Since  the 
mariners  were  not  familiar  with  these  specific  devices,  they  were  instructed  in  the  use  of  the 
features  and  functions  relevant  to  that  device.  Figures  C-1  through  C-3  present  summaries 
of  the  OSL  device  manipulation  procedures  as  well  as  the  features  and  functions  available. 
Figures  C-4  through  C-6  summarize  the  corresponding  capabilities  for  the  Robertson 
device. 
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Familiari/ation  with  all  components  of  the  experiment  Each  mariner  was  then  familiarized 
with  the  operations  of  the  full-mission  shiphandling  simulator  and  other  bridge  equipment 
(such  as  the  operation  of  the  radar/ARPA  and  the  Engine  Order  Telegraph  and  the  location 
of  radios,  telephones,  general  alarms,  and  ship's  whistle).  In  addition,  he  was  provided 
with  a  description  of  the  ship  model  to  be  used,  including  vessel  dimensions  and 
speed/RPM  tables  (see  Tables  B-2  and  B-3  of  Appendix  B).  A  familiarization  scenario  was 
then  run  (inbound  New  York,  coastal  to  pilot  station,  clear  visibility,  no  traffic)  so  that  he 
could  "get  the  feel"  of  the  integration  of  the  ECDIS  with  the  simulator,  of  the  ship  model, 
of  the  procedures,  and  of  nuances  in  the  visual  scenes.  During  this  familiarization,  a 
member  of  the  research  staff  remained  on  the  bridge  to  help  him  if  necessary. 

Testing  on  the  first  ECDIS  device.  Following  the  familiarization  run,  the  mariner  was 
asked  to  perform  various  functions  on  the  ECDIS  and  a  checklist  was  used  to  determine  his 
proficiency  and  any  re-training  he  might  need.  When  the  mariner  was  deemed  proficient  in 
the  use  of  the  ECDIS,  the  experimental  scenarios  using  that  device  with  variations  in  mode 
were  then  run.  Figures  C-7  and  C-8  present  the  training  objectives  checklist  used  to 
evaluate  the  mariner's  understanding  of  the  two  devices. 

Experimental  runs  with  the  first  device.  Prior  to  the  start  of  each  experimental  run,  the 
mariner  was  provided  with  charts  of  the  area,  a  summary  of  the  visibility  conditions,  the 
wind,  the  currents,  and  the  time  of  day  in  which  his  watch  was  to  start .  He  was  then 
asked  to  prepare  a  detailed  passage  plan.  (ECDIS  was  not  used  for  this  planning,  except 
for  one  brief  exercise  with  the  Robertson  device.)  During  the  series  of  runs  with  the  first 
experimental  ECDIS,  he  also  made  one  of  the  no-ECDIS  baseline  runs. 

The  second  device  and  the  remaining  runs.  When  the  series  of  simulator  runs  with  one 
device  was  completed,  the  ECDIS  was  changed  and  another  period  of  training  specific  to 
the  second  device  was  initiated.  The  remaining  simulator  runs  with  the  second  ECDIS  in 
aU  its  modes  and  the  remaining  baseline  scenario  were  then  completed. 

Data  collection  during  the  nms.  Data  were  collected  during  route  monitoring  activities  both 
by  the  simulation  computer  and  by  experimenters  at  the  remote  Human  Factors  Station 
(HFS).  The  computer  provided  automatic  recordings  of  ownship  information  (such  as 
heading,  rudder  angle,  and  down-track  and  cross-track  distance).  At  the  same  time  from 
the  HFS,  video  recordings  were  made  of  bridge  activity  and  tallies  were  kept  of  specific 
ECDIS  feature/function  use.  ’Track-plots”  were  periodically  printed  from  the  Visual 
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Situation  Display  to  provide  another  record  of  the  mariner's  performance  during  the  transit 
In  addition,  expert  evaluation  was  conducted  for  a  judgment  of  the  mariners  overall 
navigation/collision  avoidance/bridge  management  performance. 

Pnst-scenario  debriefing.  At  the  conclusion  of  each  experimental  scenario,  a  focused 
debriefing  session  was  conducted,  lasting  approximately  30-45  minutes.  Data  were 
collected  from  the  mariner  via  rating  scales  and  questionnaires,  to  assess  his  own 
performance  on  a  variety  of  measures,  including  situational  awareness,  operator  workload, 
safety,  and  specific  ECDIS  features  and  functions  they  used/wanted.  In  addition,  they 
were  afforded  the  opportunity  to  discuss  any  aspects  of  the  specific  scenario  and/or  the 
simulation  in  general. 

Post-experiment  questionnaire.  Following  the  completion  of  all  test  scenarios,  the 
mariner's  week  concluded  with  an  extensive  post-experiment  debriefing.  This 
questionnaire  asked  the  mariners  for  their  professional  opinions  and  assessments  on  a  wide 
range  of  ECDIS  related  issues,  including  the  overall  test  program,  charted  features  and 
functions  for  safe  ECDIS  route  monitoring,  the  chart  display,  the  presentation  of  radar 
features,  and  use  of  ECDIS  in  the  "real  world". 

C  4  OBSERVATIONS  ON  TRAINING  THE  MARINERS  IN  THE  USE  OF  THE 
ECDIS  SYSTEMS 

A  number  of  inter-related  training  issues  can  be  identified.  The  first  is  the  pperatiQ.n  of  the 
■Specific  device.  Initial  training  with  the  specific  device  was  essential  for  all  six  of  the 
mariners.  Because  the  user  interfaces  on  the  two  devices  were  so  different  from  each 
other,  training  on  the  second  device  was  as  necessary  as  training  on  the  first.  (Note  that 
several  of  the  mariners  also  reported  interface  difficulties  with  the  unfamdiar  ARP  A  on  the 
MSI/CAORF  bridge.) 

MSI/CAORF  staff  identified  another  potential  need  for  training.  The  pre-experiment 
questionnaire  and  subsequent  discussions  showed  that  not  all  of  these  experienced  mariners 
had  meaningful  knowledge  of,  or  experience  with,  DGPS.  For  the  safety  of  navigation, 
mariners  can  not  be  totally  dependent  on  the  reliability  and  accuracy  of  DGPS  or  on  built-in 
safeguards  and  alarms  in  ECDIS  systems.  Training  on  the  nature  of  DGPS  and  the 
device’s  use  of  this  sensor  should  be  included  in  any  ECDIS  training.  Some 
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understanding  of  the  electronic  chart  is  appropriate  as  well.  This  is  especially  important 
for  an  ECDIS  system  that  allows  a  ship’s  crew  to  digitize  their  own  charts. 

The  participating  mariners  were  asked  if  they  felt  that  any  “special  training”  was  needed 
Five  of  the  six  participating  mariners  expressed  the  view  that  experienced  mariners  should 
need  training  only  on  the  specific  device  to  be  used.  They  felt  that  more  general  training 
and  certification  are  not  needed.  Only  one  individual  expressed  the  opinion  that  a  certified 
course  to  obtain  an  endorsement  was  needed. 
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OSL  MENU  SUMMARY 


Radar  Display/Gain;  _ _ _ 

Chart  1  &2:  In/Out  (or  choose  value  to 

_  adjust  scale)  _ _ _ _ 

Markers!  Compass  Rose 

North  Arrow 

_ Bearing  Line _ _ 

Charts  Alpha:  Prev.  Box 

Next  Box 

Enlarge 

Vessel 

X-Track 

Station 

Line 

Tgt  Position 

_ Cursor _ _ _ 

Position :  Change  Positioning  Device 

Adjust  Offset  TrackBall  Adjust 

Set  Offset  to  0. 

Freeze  Position  Plan 

_ Position  Log _ _ 

Layouts:  Docking 

Local  Nav. 

Extended  Nav. 

_ Split  Screen _ _ _ 

Units:  Coordinate 

Distance 

_ Speed _ _ _ 

Lines:  Edit  Lines  Display  Set 

Change  Set  Name 
Select  Set  Color 
Reverse  Direction 
Delete  line 
Add  line 
Lay  Waypoint 
Attach  to  Waypoint 

_  Change  X-Track 

Stations  Radar:  Drop  Target 

Test  Pattern 
Trigger  Calibrate 
Azimuth  Calibrate 
Gyro  Dela' 


Figure  C- 1 .  OSL  Menu  Summary 


OSL  NAVIGATION  SUMMARY 


1.  Alpha  Boxes 

A.  Select  ALPHA  from  the  main  menu. 

B.  Current  box  is  highlighted  in  red.  The  contents  of  that  box  are  highlighted  on 

submenu. 

2.  To  change  CHART  and  CHART  SCALE 

A.  Select  CHART  from  the  main  menu.  Then  select  the  number  of  the  chart  you 

want  from  the  chart  menu  (Note:  make  sure  correctstation  file  is  displayed  - 
go  to  STATION  on  main  menu) 

B.  To  change  chart  scale,  select  the  desired  scale  from  the  control  grids  marked 

CHART  1  or  CHART  2  (Note:  Chart  2  may  not  be  present  depending 
on  which  Layout  is  chosen). 

3.  To  enahle/disahle  RADAR  OVERLAY 

A.  Press  the  box  on  control  panal  marked  RADAR.  Radar  overlay  is  turned  on 

when  box  is  highlighted.  Press  again  to  Disable. 

B.  Increase/Decrease  radar  gain  by  sliding  finger  along  RADAR  GAIN 

4.  Chart  Attributes 

A.  To  turn  on/off  chart  attributes,  press  MARKERS  on  main  menu.  Then  select 

attributes  you  want  (COMPASS  ROSE,  NORTH  ARROW,  BEARING 
LINE). 

B.  To  change  chart  layout,  select  LAYOUT  from  main  menu,  then  choose  your 

desired  layout  (DOCKING,  LOCAL  NAV,  EXTENDED  NAV,  SPLIT 
SCREEN) 

5.  Manually  Adjust  Ownship  Position 

A.  select  POSITION  from  main  menu,  then  select  ADJUST  OFFSET.  To 

place  the  ship  where  the  cursor  currently  is,  select  MOVE  TRACKBALL 
TO  ADJUST.  To  undo,  select  SET  OFFSET  TO  0. 

6.  To  Acquire  Radar  Targets 

A.  Place  cursor  over  target  to  be  acquired.  Press  the  middle  button  on  trackball 

panel  to  lock  on. 

B.  Target  information  will  be  displayed  in  TARGET  POSITION  box. 

C.  To  drop  an  acquired  target,  select  RADAR  from  main  menu,  then  select 

DROP  TARGET 

7.  Take  RANGE  and  BEARING 

A.  Place  cursor  over  selected  target.  RANGE  and  BEARING  from  ownship  are 
displayed  in  ALPHA  BOX  (CURSOR). 


Figure  C-2.  OSL  Navigation  Summary 
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OSL  FEATURES  AND  FUNCTIONS 


BUTTON  FUNCTION 


Radar  Display:  The  radar  display  will  be  overlayed  on  the  screen  when  this  is 
pressed. 

Radar  Gain  By  moving  your  finger  from  left  to  right  on  the  bar,  the  radar  gain  is 
increased,  and  it  is  decreased  by  sliding  your  finger  from  right  to  left 
Chart  1  &2  These  control  grids,  are  used  to  select  the  scale  which  you  desire  for  the  two 
chart  displays.  Chart  one  is  the  primary  chart  on  the  left,  and  chart  two  is 
the  chart  on  the  right.  This  chart  may  not  be  present  depending  on  the 
layout  that  is  being  used. 

In  Hitting  In,  will  zoom  in  one  spot  on  the  scale,  each  time  it  is  pressed. 

Out  Pressing  Out,  zooms  out  in  the  same  fashion  as  In  works. 

Markers  Compass  Rose:  Selecting  this  will  light  a  compass  rose  on  the  primary 
chart. 

North  Arrow:  The  North  arrow  will  light  up  in  the  top  right  comer 
of  the  primary  chart,  when  this  is  hit. 

Bearing  Line:  The  bearing  line,  is  a  solid  line  from  the  own  ship  symbol 
to  the  cursor,  and  works  much  the  same  as  an  EBL. 


Charts  Select  the  chart  you  desire  to  use,  just  as  you  would  select  a  paper  chart. 

Then  hit  the  button  that  has  the  correct  chart  number  on  it. 

Alpha  This  control  grid  is  used  to  select  what  is  (hsplayed  in  the  individual 

information  boxes.  They  are  found  either  on  the  right  side  of  the  screen,  or 
on  the  bottom  of  the  screen,  depending  on  the  layout  that  has  been 
selected.  A  red  outline  will  be  around  the  box  that  is  currently  ready  for 
changing. 

Prev.  Box:  This  button  moves  the  red  outline  back  to  the  previous  box. 
Next  Box:  Hit  next  box,  and  the  next  box  over  will  be  outlined,  for  an 
information  update. 

Enlarge:  Enlarge,  will  make  the  important  information  in  the  selected 
box  larger,  and  easier  to  read. 

Vessel:  The  vessel  information  selection,  will  give  the  position  of  the 
ship. 

X-Track:  This  information  box  gives  cross  track  information. 

Station:  Station  gives  buoy  information,  for  a  selected  buoy  or  mark. 
Line:  Line  gives  waypoint  information,  including  position,  course  to 
steer,  and  distance  between  waypoints. 

Tgt  Position:  Target  position,  displays  ARPA  information  on  selected 
radar  targets. 

Cursor:  The  cursor  information  box  gives  range  and  bearing  from  ship, 
along  with  Lat.  and  Long,  of  cursor  position. 

Position  Change  Position  Device:  Selects  which  navigation  device  you  desire 
to  use  to  update  your  position.  For  CAORF.  select  NMEA  0183.  the  othex 
choices  will  not  work. 


Figure  C-3.  OSL  Features  and  Functions 
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Layouts 


Units 


Lines 


Stations 


Radar 


Adjust  Offset:  To  place  the  ship  where  the  cursor  presently  is,  hit  Move 
Trackball  to  adjust,  and  the  ship  will  move  to  that  spot.  To  undo 
hit  Set  Offset  to  0. 

Freeze  Position  Plan:  Freezes  position,  for  easier  writing  down  of 
information. 

Playback/Clear  Position  log:  Will  be  used  in  the  future,  but  currently 
is  not  functioning. 

Docking:  This  layout  of  the  OSL  will  display  Chart  one  only  on  the 
screen,  without  any  information  boxes. 

Local  Nav.:  Again  this  display  will  only  have  the  primary  chart,  but  it 
will  have  seven  information  boxes  on  the  right  side  of  the  screen. 

Extended  Nav.:  The  primary  chart  will  be  displayed  along  with  a  small 
chart  #2  in  the  top  right  comer,  with  four  information  boxes  underneath. 
Split  Screen:  The  split  screen  layout  will  display  charts  one  and  two, 
side  by  side,  and  of  equal  size,  with  six  information  boxes  underneath. 

Coordinate:  Choose  from  Lat.  and  Long.,  Universal  Time  Meridian, 
Rectangular  coordinates,  and  Polar  coordinates. 

Distance:  Select  distance  in  Meters,  Feet,  Cables,  Miles  (statute),  and 
Nautical  Miles. 

Speed:  Ships  speed  can  be  either  in  Knots,  meters  per  second,  or 
Kilometers  per  hour. 

Edit  Lines:  Edits  and  creates  route  plan  that  you  select. 

Display  Set  must  be  lit  in  order  for  you  to  see  the  route  on  the  screen. 
Change  Set  Name  is  used  to  change  the  name  of  a  route  plan. 

Select  Set  Color  lets  you  choose  the  color  of  the  lines  of  your  plan. 
Reverse  Set  Direction  will  give  the  reciprocal  courses  for  any  plan. 
Delete  line  erases  the  line  that  the  cursor  is  on. 

Add  line  creates  a  waypoint,  by  putting  the  cursor  in  the  desired  spot,  then 
hitting  Lay  Waypoint,  then  go  back  to  the  previous  waypoint  and  hit 
Attach  to  Existing  Waypoint  and  a  line  will  connect  the  two. 

Change  X-Track:  Choose  which  route  plan  you  desire  to  use  with  this 
function. 

Stations  are  the  buoy  files,  in  order  to  display  the  correct  buoys  for  the 
chart  being  used,  use  the  last  three  digits  of  vour  chart  number  and  select 
all  the  stations  with  that  number.  There  is  one  station  for  each  color  group 
of  buoys. 

Drop  Target:  Drops  the  acquired  target  from  the  ARP  A. 

Test  Pattern:  Gives  a  radar  test  pattern  similar  to  an  archery  target. 
Trigger  Calibrate:  Do  not  touch,  or  radar  overlay  will  be  incorrect. 
Azimuth  Calibrate:  Same  as  above. 

Gyro  Delay:  Same  as  above. 


Figure  C-3.  OSL  Features  and  Functions  (cont.) 
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ROBERTSON  MENU  SUMMARY 
FOR  ROUTE  MONITORING 


rilRSOR  FfTNCTTONS  IN  MONITOR  MODE; 

No  Cursor  Nav-Lines  ARPA  Targets 

Manual  Center  Range/Bearing  DR-Reset 


MAIN  MENU 

Chart 

SITB-MENU 

New  Scale . 

True  Motion 

Relative  Motion 

Auto  Center 

Select  Level/Scale 

Ship 

Manual  Input . 

Vector  Qn/Off 

Vector  Time 

Track  Time  On/Off 

Labels  On/Off 

Input  Speed 

Input  Course 

Input  Drift 

Input  Set 

Reset  DR-Pos 

Echo  Sounder 

Open  Log . 

Delete 

Backup 

Distance 

Draft 

Create  Log 

ANTS 

Enable  ANTS 

Change  Route 

POS 

CTS 

XTE 

WOW 

ARPA 

Enable  Targets . 

Vectors  On/Off 

True/Relative 

Nav 

Dead  Reckoning 

Decca 

Transit 

Loran-C 

GPS 

KF 

Setup 

Bright  Control . 

Bright  Palette 

Normal  Palette 

Dusk  Palette 

Night  Palette 

Set  Clock 

Ground  Time 

Update  System 

Contrast-Text 

Brighmess-Text 

Contrast-Graphic 

Keyboard  Backlight 

Figure  C-4.  Robertson  Menu  Summary  for  Route  Monitoring 
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ROBERTSON  NAVIGATION  SUMMARY 


FOR  CURSOR  FUNCTIONS  IN  THE  MONITOR  MODE,  USE  THE  FUNCTION 
KEY  TO  SELECT  OPTIONS 

1.  NO  CURSOR:  Used  to  turn  the  cursor  OFF 

2.  NAV-LINES:  Used  to  measure  distance  between  two  points. 

3.  ARPA  TARGETS  :  Used  to  display  information  about  a  target 

To  display  target  information:  Place  the  cursor  over  the  selected  target,  press 
SET  key  once,  and  that  target  information  will  be  displayed  in  upper  right  of  text 
screen 

4.  MANUAL  CENTER:  Used  to  move  the  chart  with  vessel  to  the  cursor  position. 

Move  the  cursor  to  the  area  on  chart  wanted  as  vessel  position  and  press  SET  Key 
once.  Chart  will  be  redrawn  with  vessel  placed  at  the  cursor's  position.  NOTE: 
THE  CURSOR  KEY  WILL  MOVE  THE  CURSOR  TO  THE  CENTER  OF  THE 
CHART. 

5.  RANGE/BEARTNG:  Used  to  measure  distance  from  ownship  to  other  objects 

(i.e.,  aids  to  navigation,  land).  Using  the  trackball,  move  the  cursor  to  a  selected 
object.  That  object's  range  and  bearing  from  ownship  will  be  displayed  on  text 
screen 

6.  DR-RESET:  Used  to  manually  re-set  ownship's  Dead  Reckoning  position 


MENU  FUNCTIONS  IN  MONITOR  MODE: 

1.  CHART:  This  function  is  used  if  chart  scale  or  center  is  changed. 

NEW  SCALE:  Choose  new  LEVEL  or  SCALE 

TRUE  MOTION:  Displays  ownship  movement  on  FIXED  chart  (vessel  speed 
determines  when  chart  is  redrawn). 

RELATIVE  MOTION:  Displays  FIXED  vessel  while  chart  moves 
AUTO  CENTER:  System  decides  where  chart  center  will  be. 

2.  SHIP:  Used  for  selection  and  input  of  navigational  data. 

VECTOR  ON/OFF:  Displays  a  vector  from  ownship 
TRACK  TIME:  Records  the  time  between  recorded  track  positions 
LABELS  ON/OFF:  Will  display  the  time  vessel  passed  recorded  track  positions 
MANUAL  INPUT:  Allows  manual  input  of  vessel  SPEED,  COURSE,  DRIFT, 
SET.  RESET  DR-POS  is  used  to  manually  update  DR  position  (lat/long). 

3.  ECHO  SOUNDER:  If  an  echo  sounder  is  interfaced  with  ECDIS,  it  is  possible  to 

log  signals  from  this  to  a  file.  THIS  FUNCTION  DOES  NOT  CURRENTLY 
WORK. 


Figure  C-5.  Robertson  Navigation  Summary 
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4.  ANTS:  ENABLE  ANTS:  Allows  for  automatic  track  steering  along  a  pre-planned 

route.  Alarms  are  sounded  when  tolerance  parameters  are  exceeded. 
CHANGE  ROUTE:  Allows  you  to  change  selected  route 
POS:  Position  difference  tolerance 
CTS:  Off-course  tolerance 
XTE:  Cross-track  tolerance 

WOW:  Wheel  over  warning  time  -  alarm  sounds  to  initiate  course  changes 

5.  ARPA:  If  an  ARPA  radar  is  interfaced  with  ECDIS,  both  fixed  and  moving  targets 

wiU  be  displayed. 

ENABLE  TARGETS:  Will  allow  acquired  targets  to  be  displayed 
VECTORS  ON/OFF:  Turns  on/off  target  heading  vector 
TRUE/RELATIVE:  Shows  either  true  or  relative  target  motion 

6.  NAV:  Allows  you  to  select  active  navigational  device:  DR,  DECC A,  TRANSIT, 

LORAN-C,  GPS.  NOTE:  in  the  event  of  losing  navigational  data  from 
selected  receiver,  ECDIS  will  switch  to  DR  position. 

7.  SETUP:  Allows  you  to  change  the  brightness  of  the  chart  display 

(Bright,  Normal,  Dusk,  Night),  contrast  and  brighmess  of  text  display, 
and  to  update  system.  If  date  or  time  in  text  display  is  incorrect,  can  be 
edited  by  using  the  SET  CLOCK  key. 


Figure  C-5.  Robertson  Navigation  Summary  (cont.) 


C-13 


ROBERTSON  FEATURES/FUNCTIONS 


BUTTON  FUNCTION 


Beacon . Creates  an  overlay  for  lighted  navigation  aids,  using  a  yellow 

symbol. 

Buoy . Turns  an  overlay  of  buoys  on  and  off. 

Marks . First  depression  of  the  button  will  show  an  overlay  of  charted 

wrecks,  the  second  will  add  on  an  overlay  of  pilings. 

Depth . Activates  depth  display  in  meters  from  chart  datum. 

Text . First  overlay  puts  names  of  geographical  areas  up,  and  the  second 

overlay  writes  information  for  the  navigation  aids. 

Grid . Displays  a  Latitude  and  Longitude  grid. 

Alarm . By  pushing  the  alarm  button,  you  can  shut  off  any  alarm  that  may 

have  been  activated.  Active  alarms  will  be  flashing  on  the  right  side 
of  the  menu  screen. 

Reset . Resets  chart  attributes  to  IMO  Standards. 

Monitor . Pushing  this  button  puts  the  Eedis  in  monitor  mode,  where  ships 

current  position  is  displayed. 

Plan . Selecting  plan  mode,  brings  up  menus  that  are  used  to  set  up 

waypoints  and  plan  routes. 

On . Turns  the  Robertson  Disc  Navigation  System  on. 

Off . There  are  two  off  buttons,  to  turn  the  machine  off,  push  both 

buttons  at  the  same  time,  and  wait  five  seconds. 

Set . Depending  on  what  function  is  currently  up  on  the  system,  the  set 

key,  used  in  conjunction  with  the  rollerball,  can  be  used  to  set 
certain  aspects,  such  as  waypoints  and  navlines. 

Cursor . The  cursor  key,  depending  on  current  function,  will  usually  center 

the  cursor  in  the  middle  of  the  screen. 

Function . Pushing  the  function  button  wiU  highlight  and  select  the  desired 

function  from  the  top  of  the  text  screen. 

Numbers  and . These  buttons  are  used  to  enter  numbers  and 

Letters  keypad  letters.  To  get  the  letter  C,  for  example,  push  the  A/B/C  button  once 

for  A,  twice  for  B,  and  three  times  for  C. 

Esc . The  escape  key  will  let  you  go  back  one  screen  from  the  present 

screen,  and  is  usually  the  only  way  to  exit  from  one  of  the  menus. 

B .  S . This  is  the  back  space  key,  which  allows  you  to  go  back  a  space 

when  entering  information  from  the  number  and  letter  keypad. 

Enter . Push  the  enter  button  when  you  have  finished  inputting  information 

from  the  number  and  letters  keypad. 

Trackball . Use  the  trackball  to  move  the  cursor  on  the  screen. 


Figure  C-6.  Robertson  Features  and  Functions 
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OSL  TRAINING  OBJECTIVES  FOR  ROUTE  MONITORING 


Use  ECDIS  to  monitor  vessel's  progress 
effectively 

using  available  features/functions. 

1st 

Try 

2nd 

Try 

Comments 

•  Demonstrate  knowlege  of  digital  readout  information 
available  (identify  contents  of  alpha  boxes) 

■ 

■ 

•  Demonstrate  knowlege  of  charts  in  system.  Change 
chart/chart  scale,  buoy  and  station  files 

■ 

■ 

•  Demonstrate  abihty  to  enable/disable  radar  overlay; 
increase/decrease  radar  gain 

■ 

■ 

•  Demonstrate  ability  to  acquire  and  track  radar  targets 

■ 

■i 

•  Demonstrate  ability  to  turn  on/off  EC  attributes  (compass 
rose. 

North  arrow,  bearing  lines). 

■ 

■ 

•  Manually  adjust  vessel's  position  with  radar  overlay 

Bi 

•  Change  screen  layouts  of  electronic  chart 

■ 

Bi 

Perform  traditional  position  checking  with 
ECDIS. 

■ 

■ 

•  Use  ECDIS  to  take  range/bearing. 

•  Select  a  target  and  display  vessel's  range  and  bearing  to  it 

ASSESSMENT 


1:  Need  to  re-train 

2:  Satisfactory  (found  solution  with  some  trial 

and  error) 

3:  Excellent  (found  solution  immediately) 


Figure  C-7.  Checklist  for  OSL  Training  Evaluation 
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ROBERTSON  TRAINING  OBJECTIVES  FOR  ROUTE  MONITORING 


Use  ECDIS  to  monitor  vessel  s  progress 
effectively  using  available  features/functions 


Demonstrate  knowlege  of  digital  readout  information 
available 


•  Change  charts  and  chart  scales 


♦  Set  chart  motion  to  desired  setting  (true  or  relative) 


•  Set  chart  center  to  desired  (autocenter  or  manual  center 
offset)  _ 


•  Respond  to  appropnate  alarm  wammgs 


•  Display  appropriate  color  palette  for  time  of  day 


•  Invoke  track  history  with  tune  labels 


•  Turn  on/off  chart  features  (buoys,  beacons,  etc.)  and 
Reset  to 

minimum  display  content 


•  Use  numeric  keypad  to  input  any  necessary  digital 
information  (e.g.,  lat/long)  and  to  backspace 


♦Enable/disable  ANTS 


Perform  traditional  position  checking  with 
ECDIS 


•  Use  ECDIS  to  take  range  and  bearing 


•  Use  Nav-lines 


Demonstrate  ability  to  cope  with  DGPS  failure. 


•  Check  for  active  navigational  device 


•  Adjust  vessel  s  position  on  EC  based  on  independent 
position  fixing. 


•  Manually  update  vessel's  course,  speed,  set,  drift,  and 
osition  (via  lat/long). 


1st  2nd 
Try  Try 


ASSESSMENT 


1:  Need  to  re-train 

2:  Satisfactory  (found  solution  with  some  trial 

and  error) 

3:  Excellent  (found  solution  immediately) 


C-  16 


APPENDIX  D.O 


SUPPLEMENTARY  MATERIAL  FOR 
SECTION  4.0  PERFORMANCE  MEASURES 


D.  1  WORKLOAD  MEASUREMENT  TOOLS 

The  mariner's  experience  of  workload  was  evaluated  using  the  NASA-TLX  (Task  Load 
Index)  subjective  rating  scale  technique  (Hart  and  Staveland,  1988).  The  NASA-TLX 
subjective  rating  scales  were  validated  and  refined  over  a  number  of  years  for  several 
experimental  tasks  ranging  from  simple  cognitive  and  manual  control  tasks  to  supervisory 
control  tasks  and  aircraft  simulation  (Hart  and  Staveland,  1988).  Weighted  workload 
scores  for  the  different  tasks  were  found  to  correlate  highly  with  objective  measurements 
relevant  to  each  task,  although  certain  of  the  dimensions  contributed  more  to  the  overall 
scores,  depending  on  the  nature  of  the  task  as  would  be  expected  (i.e.,  certain  dimensions 
were  judged  to  be  more  important  for  certain  tasks).  In  addition,  statistically  significant 
differences  in  an  objective  measure  of  task  performance  often  coincided  with  significant 
difference  in  the  workload  scores. 

Six  workload  components  are  rated  in  the  NASA-TLX  workload  measurement  technique. 
Figure  D-1  presents  the  definitions  for  each,  and  Figure  D-2  presents  the  actual  ratings 
scales  used  for  the  study.  To  compute  the  workload  score,  each  dimension  was  weighted 
using  a  procedure  described  in  Figure  D-3.  Note  that  separate  ratings  were  obtained  for 
navigation,  collision  avoidance,  and  bridge  management.  The  breakdown  given  to  the 
mariner  of  the  types  of  tasks  falling  into  each  category  is  presented  in  Figure  D-4. 

Finally,  each  mariner  was  asked  to  describe  scenario  events  that  produced  very  high 
levels  of  workload  or  stress,  using  the  "Event  Workload"  form  given  in  Figure  D-5.  The 
main  workload  analyses  are  discussed  in  the  main  report,  as  are  comments  made  by  the 
mariners. 

D.2  THE  MEASUREMENT  OF  SITUATIONAL  AWARENESS 

The  situational  awareness  measurement  procedure  used  in  this  study  was  adapted  from 
Aretz  (1989)  and  centered  around  a  series  of  three  or  four  chartlets  per  scenario  which 
showed  the  coasthne,  aids-to-navigation  arrangement,  and  planned  route  for  the  area  just 
transited.  One  chartlet  had  a  relatively  small  scale  and  showed  the  entire  route,  while  the 
other  chartlets  showed  segments  of  the  route  at  a  larger  scale.  The  mariner  first  drew  in 
the  path  of  the  vessel  (from  memory)  with  respect  to  the  planned  trackline,  using  the 
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larger  scale  chartlets.  He  was  encouraged  to  discuss  his  progress  along  the  route,  which 
often  provided  insight  into  his  actions  ,  and  which  were  recorded  by  the  experimenter. 

The  concept  of  situational  awareness  was  then  discussed  with  the  mariner.  This 
discussion  focused  on  a  set  of  clues  or  warning  signs  that  may  indicate  a  loss  of 
situational  awareness,  which  were  identified  by  Schwartz  (1989)  in  relation  to  aircraft 
flight  and  adapted  for  use  in  this  study.  These  clues  include  ambiguity,  distraction, 
confusion,  improper  lookout  or  trackkeeping,  failure  to  comply  with  passage  plan,  and 
violation  of  basic  rules  or  procedures  including  communications  protocol. 

The  mariner  was  then  given  a  situational  awareness  rating  scale  ranging  from  1  (very 
low)  to  5  (very  high).  Using  the  smallest  scale  chartlet,  he  drew  hatch  marks  to  divide  the 
transit  into  as  many  sections  as  necessary  to  reflect  his  changes  in  situational  awareness, 
and  assigned  a  one  to  five  rating  to  each  section.  The  rating  scale  and  a  sample  chartlet 
are  presented  in  Figures  D-6  and  D-7,  respectively.  The  percent  of  total  transit  distance 
(transits  placed  end  to  end  and  distances  summed)  the  mariner  reported  spending  at  each 
of  the  five  levels  of  awareness  was  compared  across  bridge  conditions.  These  data  are 
presented  in  Table  D-1,  and  show  that  in  the  ECDIS  with  Automatic  Positioning 
scenarios  (Scenarios  1, 2, 4, 5,  and  7)  the  mariners  reported  spending  a  total  of  78.46 
percent  of  the  transit  distance  at  an  awareness  levels  of  four  or  five,  but  only  67.49 
percent  at  levels  four  or  five  in  the  Conventional  Bridge  scenarios  (Scenarios  8  and  9). 
Moreover,  they  reported  spending  only  53.58  percent  of  the  total  transit  distance  at  levels 
four  or  five  in  the  ECDIS  without  Automatic  Positioning  scenarios  (Scenarios  3  and  6). 
Thus,  these  data  show  a  generally  increased  reported  level  of  situational  awareness  with  a 
functioning  ECDIS  on  the  bridge.  Further  statistical  analyses  were  not  performed. 
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WORKLOAD 

RATING  SCALE  DEFINITIONS 


MENTAL  DEMAND 

PHYSICAL  DEMAND 

TIME  DEMAND 

OWN  PERFORMANCE 

OVERALL  EFFORT 

FRUSTRATION  LEVEL 


How  much  mental  and  perceptual  activity  was 
required  (e.g.,  thinking,  deciding,  calculating, 
remembering,  looking,  searching,  etc.)?  Was 
the  task  easy  or  demanding,  simple  or  complex, 
exacting  or  forgiving? 


How  much  physical  activity  was  required  (e.g., 
pushing,  pulling,  turning,  controlling,  activating, 
etc.)?  Was  the  task  easy  or  demanding,  slow  or 
brisk,  slack  or  strenuous,  restful  or  laborious? 

How  much  time  pressure  did  you  feel  due  to  the  rate 
or  pace  at  which  the  tasks  or  task  elements  occurred? 
Was  the  pace  slow  and  leisurely  or  rapid  and  frantic? 


How  successful  do  you  think  you  were  in  accomplishing 
the  goals  of  the  task  set  by  the  experimenter  (or 
yourself)?  How  satisfied  were  you  with  your 
performance  in  accomplishing  these  goals? 

How  hard  did  you  have  to  work  (mentally  and 
physically)  to  accomplish  your  level  of  performance? 

How  insecure,  discouraged,  irritated,  stressed  and 
annoyed  versus  secure,  gratified,  content,  relaxed 
and  complacent  did  you  feel  during  the  task? 


Figure  D-1.  NASA-TLX  Workload  Rating  Scale  Definitions 


MARINER  # _  RUN  #  _  SCENARIO  # 

WORKLOAD  RATINGS 


1 .  During  your  last  transit,  what  percentage  of  your  time  was  spent  on  navigation, 
collision  avoidance,  and  bridge  management  tasks? 


NAVIGATION 

COLUSION  AVOIDANCE 

BRIDGE  MANAGEMENT 

% 

% 

% 

2.  Piace  an  "X"  on  each  of  the  six  scales  to  indicate  your  experience  of  navigation  tasks 
during  your  last  transit. 


FOR  NAVIGATION  TASKS: 

MENTAL  DEMAND  LCW| — I — | — I — | — I — | — I — \ — 1 — j — I — j — I — j — I — ] — I — | — I — |hiGH 

PHYSICAL  DEMAND  l£W] — I — | — I — | — I — | — I — | — I — | — I - 1 — I — ] — I — | — I — ] — I — |hIGH 

TIME  DEMAND  LCW| 1 j 1 j 1 1 1 1 1 1 1 1 1 1 1 1 1 j 1 [HIGH 

OWN  PERFORMANCE  GOCO| - 1 — j — I — | — 1 — \ — I — | — ! — | — I - 1 — 1 — | — I — | — I — | — I — |  POOR 

OVERALL  EFFORT  LCW| — I — j — I — | — I — ] — i — | — i 1 1 1 — i — | — I — ] — I — j — I — |  HIGH 

FRUSTRATION  LEVEL  LCW| — I — j — l — | — l — | — I — | — ! — | - 1 - \ — I — | — I — | — I — | — I — |hiGH 


Figure  D-2.  Workload  Rating  Scale  Questionnaire 
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MARINER  it _  RUN  #  _  SCENARIO  it - 

3.  Place  an  "X"  on  each  of  the  six  scales  to  indicate  your  experience  of  collision  avoidance  tasks 
during  your  last  transit. 

FOR  COLLISION  AVOIDANCE  TASKS: 

MENTAL  DEMAND  ICW| — I — | — I — | — i — |— • — | — I — j — ' — | — ' — \ — • — |  •  1  ' 

PHYSICAL  DEMAND  ICWj — I — j — I — ] — I — | — I — \ — ' — 1 — ' — | — ' — 1 — ' — 1 — • — 1 — ^ 

TIME  DEMAND  lOW| — i — ] — | — | — i — j — l — | — I — | — I — } — I — | — I — | — • — | — 1^ — 

OWN  PERFORMANCE  G0CD| — I — \ — I — j — I — | — • — \ — ' — | — ' — 1 — ' — | — ' — | — * — I — '  1^°°^ 

OVERALL  EFFORT  lOWj — I — | — I — | — I — | — I — | — • — J — I — ] — ' — | — ' — | — ' — | — '  |  HIGH 

FRUSTRATION  LEVEL  ICW| — I — | — I — | — I — j — ' — ] — ' — \ — • — 1 — ' — I — ' — | — * — j  '  I  HIGH 

4.  Place  an  "X"  on  each  of  the  six  scales  to  indicate  your  experience  of  bridge  management  tasks 
during  your  last  transit. 

FOR  BRIDGE  MANAGEMENT  TASKS: 

MENTAL  DEMAND  lOW| — I — | — I — \ — I — \ — I — \ — ' — 1 — « — 1 — ' — 1 — ' — 1  ‘  I  '  IhIGH 

PHYSICAL  DEMAND  LCW| — I — | — ! — j — I — | — I — | — > — | — ' — 1 — ' — | — ' — 1 — ' — 1 — '  |HIGH 
TIME  DEMAND  lOW| — i — | — | — j — l — j — I — j — I — | — I — \ — I — \ — ' — | — ' — j — ' — |H'GH 

OWN  PERFORMANCE  GDCOj — I — | — I — ] — I — | — I — [ — ' — | — ' — | — ' — 1 — ' — | — ' — I  '  1^"°^ 

OVERALL  EFFORT  ICW| — I — | — I — | — I — j — I — j — I — | — ' — \ — ' — 1 — ' — | — ' — | — '  |  HIGH 

FRUSTRATION  LEVEL  ICW) — I — \ — I — \ — I — | — ' — \ — • — | — ' — j — ' — | — ' — 1 — ' — 1  '  IhIGH 

Figure  D-2.  Workload  Rating  Scale  Questionnaire  (cont.) 


D-5 


Sources  of  Workload  Comparisons 

Instructions:  For  each  of  the  following  pairs  of  rating  scale  titles,  circle  the  member  of 
each  pair  you  feel  was  tiie  more  important  contributor  to  your  experience 
of  workload  during  navigation  (either  with  the  BCD  IS  or  paper  chart). 


Overall  Effort  /  Own  Performance 
Time  Demand  /  Overall  Effort 
Own  Performance  /  Frustration 
Physical  Demand  /  Own  Performance 
Frustration  /  Overall  Effort 
Performance  /  Time  Demand 
Mental  Demand  /  Physical  Demand 
Frustration  /  Mental  Demand 


Time  Demand  /  Frustration 
Physical  Demand  /  Frustration 
Physical  Demand  /  Time  Demand 
Time  Demand  /  Mental  Demand 
Own  Performance/  Mental  Demand 
Mental  Demand/  Overall  Effort 
Overall  Effort/  Physical  Demand 


FOR  USE  BY  STAFF 


SCALE  TITLE 

TALLY 

WEIGHT 

MENTAL  DEMAND 

PHYSICAL  DEMAND 

TIME  DEMAND 

OWN  PERFORMANCE 

OVERALL  EFFORT 

FRUSTRATION 

Total  Count= 


NASA -TLX  Weighting  Procedure 

To  determine  the  overall  workload  score  for  each  of  the  three  sets  of  ratings  per  scenario 
(navigation,  collision  avoidance,  and  bridge  management),  each  dimension  is  weighted  in 
terms  of  its  relative  importance  to  the  individual's  experience  of  workload.  At  the  end  of 
the  week  when  both  devices  had  been  used  and  at  least  one  of  the  baseline  scenarios  was 
completed,  the  mariner  made  the  above  workload  comparisons.  He  was  instructed  to 
circle  one  member  of  each  of  the  15  pairs  shown  above  to  indicate  which  of  the 
dimensions  was  the  more  important  contributor  to  his  experience  of  workload.  The 
frequency  with  which  each  dimension  was  chosen  (0-5  times)  then  served  as  that 
dimensions  weight  to  be  applied  to  the  workload  score.  One  set  of  workload  dimension 
weights  was  obtained  from  each  mariner,  and  applied  to  all  three  sets  of  workload  scores, 
producing  an  overall  workload  score  for  each  of  the  three  task  categories. 


Figure  D-3.  Workload  Comparisons  and  Weighting  Procedure 
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GUIDE  TO  BRIDGE  TASK  BREAKDOWN 


NAVIGATION 

1.  Track  monitoring 

a.  Position 
determination 

b.  Awareness  of 
vessel's  position 
relative  to  aids  to 
navig.,  channels, 
dangers  to  navig. 


2.  Control  of  course/speed 


3.  Knowledge  of  state  of 
tides,  currents,  weather 
conditions  affecting 
navigation. 


4.  Monitor  passage  plan  for 
required  course  changes 
&  other  actions.  _ 


COLLISION 

AVOIDANCE 

1.  Maintain  efficient 

lookout:  visual,  aural,  & 
radar/ARPA 


2.  Monitor  traffic  for  risk 
of  collision  and  take 
appropriate  actions  to 
avoid  close  quarters 
situations. 

3.  Intership  (  VHP) 
Communications 


BRIDGE 

MANAGEMENT 

1.  Integration  of 

engineroom  &  other 
dept,  activities  relative 
to  safe  navigation, 
(preparations  for  pilot, 
anchoring,  docking,  etc 


2.  Maintain  logs,  other 
administrative  duties. 


3.  Ship  to  shore  and 
Intraship 
Communications 


Figure  D-4.  Mariner's  Guide  to  Bridge  Task  Breakdown 
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MARINER  # _  RUN  #  _  SCENARIO  # _ 

EVENT  WORKLOAD  RATINGS 

1.  in  your  judgement,  what  event(s)  during  your  last  transit  produced  the  greatest  workload  level? 
Please  describe  one  event  and  the  way  you  handled  it  in  the  space  provided  below.  Use  a 
separate  "Event  Workload  Ratings'  form  to  describe  each  other  event. 


2.  Place  an  "X”  on  each  of  the  six  scales  to  indicate  your  experience  during  this  event. 

MENTAL  DEMAND  LCW| — I — ] — I — | — 1 — ] — I — | — 1 — | — I — | — I — \ — I — | — I — | — I — |hIGH 

PHYSICAL  DEMAND  L£3W| — I — | — I — ] — I — | — I — | — I — | — 1 — | — I — | — I — j — I — | — I — |h1GH 
TIME  DEMAND  lOW| — | — j — | — | — | — | — i — | — i — | — | — | — | — | - ] — j — | — | — \ — [HIGH 

OWN  PERFORMANCE  QXO| — I — | — I — | — I — ] — I — | — I — | — I — | — ! — | — I — | — 1 — | — I — |POOR 
OVERALL  EFFORT  lOW| — I — j — i — | — i — | — i — ] — i — | — | — | — \ — | — I — J — l — | — I — |  HIGH 
FRUSTRATION  LEVEL  LOW| — I — j — ! — | — I — | — I — | — I — | — I — | — I — ] — I — j — 1 — | — I — |H!GH 

3.  Please  rate  your  level  of  navigational  safety  during  this  event: 

1 - 1 - 1 - 1 - 1 

1  2  3  4  5 

VERY  SAFE  HIGH  RISK 

4.  How  do  you  think  the  ECDIS  influenced  your  performance  during  this  event? 

Figure  D-5.  Event  Workload  Rating  Scale  Questionnaire 
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Table  D- 1 .  Percent  of  total  transit  distance  across  all  mariners 

spent  at  each  level  of  reported  situational  awareness. 


Situational  Awareness  Level 

5 

4 

3 

2 

1 

Bridge  Condition 

highest 

lowest 

ECDIS  w/  Auto  Update 

33.94 

44.52 

14.94 

5.92 

0.67 

Conventional  Bridge 

32.5 

35.0 

19.72 

8.83 

3.96 

ECDIS  w/  Manual  Update 

23.21 

30.38 

29.61 

16.81 

0.0 

In  addition  to  examining  the  length  of  the  transits  the  mariners  reported  spending  at  each 
level  of  situational  awareness,  these  ratings  were  used  in  a  correlation  analysis  with  path 
re-creation  errors.  The  mariner's  hand  drawn  path  of  the  vessel  was  visually  compared 
with  the  actual  path  of  the  vessel  as  recorded  in  Visual  Situation  Display  (VSD)  plots, 
including  both  his  left/right  placement  of  own  ship  with  respect  to  the  planned  trackhne, 
and  his  placement  of  course  changes  in  relation  to  actual  course  changes.  This  analysis 
was  chosen  because  it  was  hypothesized  that  the  errors  made  in  re-creating  the  ship’s  path 
would  tend  to  be  located  in  sections  of  the  transit  where  the  mariner  indicated  a  lower 
level  of  situational  awareness.  This  analysis  can  be  interpreted  as  an  estimate  of  the 
mariner’s  level  of  situational  awareness.  In  general,  the  average  number  of  path  re¬ 
creation  errors  was  fairly  small  and  the  variability  was  high.  There  was  an  average  of  4.5 
errors  (SD  =  1.72)  for  the  Conventional  Bridge  scenarios,  an  average  of  2.54  errors  (SD  = 
2.18)  for  the  ECDIS  with  Automatic  Positioning  scenarios,  and  an  average  of  4.56  errors 
(SD  =  2.19)  for  the  ECDIS  without  Automatic  Positioning  scenarios.  Overall,  a  very 
small  negative  correlation  was  found  which  indicated  that  a  larger  number  of  path  re¬ 
creation  errors  was  slightly  associated  with  lower  levels  of  reported  situational  awareness 
at  the  location  of  the  error,  and  the  statistical  significance  of  this  correlation  is  only 
marginal  (Correl.  =  -  0.126,  p  <  0.12). 

This  correlation  was  examined  further  by  breaking  up  the  transits  into  segments 
corresponding  to  the  legs  given  in  Question  4  of  each  post-scenario  questionnaire  (This 
question  asked  the  mariner  to  indicate  both  his  primary  and  secondary  method  of 
navigation  for  each  leg  of  the  transit.).  For  each  transit  leg,  then,  there  was  an  indicated 
primary  method  of  navigation,  a  situational  awareness  rating,  and  some  number  of 
associated  path  re-creation  errors  (zero  or  more).  The  relation  between  path  re-creation 
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STTTTATTONAL  AWARENESS  RATING  SCALE  AND  SAMPLE  CHARTLETS 


Situational  awareness  is  defined  as  an  accurate  perception  of  the  factors  and  conditions 
that  effect  the  vessel  and  the  bridge  team  during  a  specific  period  of  time.  In  other  words, 
you  know  what  is  going  on  arotmd  you.  A  high  level  of  situational  awareness  yields  a 
decreased  exposure  to  risk. 

We  request  that  you  rate  your  situational  awareness  over  the  length  of  the  last  transit.  To 
assist  you  in  the  evaluation  of  your  situational  awareness,  please  note  the  following 
indicators  or  warning  signs  of  a  loss  of  situational  awareness. 

APPARENT  AMBIGUITY  -  two  or  more  independent  sources  of 
information  did  not  agree. 

DISTRACTION  -  you  got  completely  focused  on  one  item  or  event 
and  excluded  all  others. 

CONFUSION  -  you  became  confused  and  anxious  about  a  particular 
situation.  You  felt  that  you  were  losing  control  of  the  vessel. 

IMPROPER  LOOKOUT  OR  TRACKEEPING  -  (as  a  result  of 
DISTRACTION)  you  failed  to  keep  a  proper  lookout,  lost  track  of 
where  the  vessel  was  relative  to  dangers  or  traffic. 

FAILURE  TO  COMPLY  WITH  PASSAGE  PLAN  -  you  failed  to 
meet  ETA's,  planned  speeds,  make  course  changes  on  time,  got  too 
far  off  track  or  other  goals  set  forth  in  passage  plan. 

VIOLATED  BASIC  RULES  -  failed  to  adhere  to  the  rules  of  the 
road,  did  not  observe  proper  communications  procedures,  failed  to 
call  vessels  .before  traffic  encounters 

Please  rate  your  situational  awareness  (SA)  using  the  procedure  described  below. 

First  draw  in  your  trackline  to  show  the  path  of  the  vessel  throughout  the  duration  of  the 
transit  on  the  larger  scale  chaitlets.  Then,  assign  a  rating  (using  the  scale  below)  to  the 
different  portions  of  the  transit  on  the  small  scale  chartlet.  You  may  divide  the  transit 
into  as  many  sections  as  necessary  to  accurately  reflect  the  changes  in  your  situational 
awareness. 


1 

2 

3 

4 

5 

very  low  SA 

very  high  SA 

Figure  D-6.  Situational  Awareness  Definition  and  Rating  Scale 
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Route  Tracking 


ISOQOO. 


errors  and  reported  level  of  situational  awareness  for  those  segments  of  the  transits  in 
which  the  ECDIS  was  used  as  the  primary  method  of  navigation  showed  a  significant 
though  small  correlation  (Correl.  =  -  0.244,  p  <  0.04).  That  is,  while  the  average  number 
of  path  re-creation  errors  was  similar  regardless  of  the  reported  primary  method  of 
navigation  (i.e.,  paper  chart,  visual,  radar/ ARPA  or  ECDIS),  when  ECDIS  was  the 
primary  navigation  method  mariners  tended  to  rate  their  level  of  situational  awareness  as 
lower  in  the  same  segments  in  which  they  made  path  re-creation  errors.  Thus,  while  this 
method  of  predicting  the  mariner's  reported  level  of  situational  awareness  from  the 
number  of  path  re-creation  errors  made  shows  some  promise,  it  needs  further  testing  and 
refinement  to  be  really  useful. 


D.3  EXPERIMENTAL  OBSERVER’ S  RATINGS  OF  MARINER  ERRORS 

To  enable  more  objective  and  reliable  scoring,  the  Experimental  Observer  had  available  a 
time  history  for  each  scenario  which  listed  the  events  that  comprised  each  scenario  in 
their  probable  order  (and  time)  of  occurrence.  The  observer  recorded  the  errors  made  by 
each  mariner  on  a  set  of  structured  tally  sheets  which  listed  the  most  probable  errors  for 
each  type  of  event. 

The  Observer  for  all  of  the  runs  holds  a  Third  Mates  License  and  has  had  two  years  of 
intensive  familiarity  with  MSI's  training  courses,  simulator  operation,  and  scenario 
design.  In  fact,  the  Observer  was  largely  responsible  for  designing  and  programming  the 
events  comprising  each  of  the  scenarios  used  in  this  study.  It  should  also  be  noted  that 
the  number  of  errors  committed  during  the  runs  is  probably  inflated  because  of  the 
artificially  high  workload  demand  placed  on  the  mariners.  It  should  be  kept  in  mind  that 
each  mariner  was  alone  on  the  bridge  (except  for  a  helmsman)  under  conditions  in  which 
they  would  normally  have  at  least  one  other  officer  present,  and  was  responsible  for  all 
aspects  of  bridge  management  and  functions. 

Table  D-2  below  presents  a  tally  of  the  types  of  errors  that  were  made  by  the  mariners. 
The  number  of  mariners  represented  in  the  tallies  for  each  scenario  is  given  at  the  top  of 
the  table.  The  table  also  gives  the  weighting  factor  assigned  to  each  error  by  expert 
mariners  during  the  development  of  this  measure  before  the  experiment  began.  As  might 
be  expected,  more  navigation  errors  occurred  overall  in  the  ECDIS  without  Automatic 
Positioning  scenarios  (Scenarios  3  and  6),  although  the  nature  of  the  most  predominant 
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Table  D-2.  Tally  of  errors  committed  during  each  scenario  for  all  mariners. 


ssss 

SCENARIO 

1 

2 

3 

4 

5 

6 

7 

8 

9 

total# 

n= 

of 

(3) 

(5) 

(3) 

(6) 

(6) 

(6) 

(6) 

(6) 

(5) 

mors 

NAVIGATION  ERRORS: 

wt 

H 

Unnecessarily  close  to  shallow  water 

(2) 

1 

HI 

Missing  channel  entrance 

(2) 

1 

HI 

Leaving  the  channel 

(2) 

1 

1 

■Ml 

Believing  that  the  ship  is  somewhere  it  isn’t, 

(1) 

1 

1 

1 

6 

1 

1 

1 

(difficult  to  determine  this) 

Striking  a  buoy 

3) 

2 

1 

HI 

Operating  on  the  wrong  side  of  the  channel, 

(1) 

6 

5 

2 

1 

H 

when  unnecessary,  for  each  three  minutes 

HI 

Dropping  anchor  in  the  wrong  anchorage 

(1) 

1 

3 

Iraorooer  use  of  radar  for  navigating 

(1) 

1 

1 

■■ 

total  #  of  errors 

7 

2 

8 

1 

1 

10 

2 

2 

6  1 

39 

COLLISION  AVOIDANCE 

ERRORS: 

Dangerously  close  to  a  collision 

(4) 

1 

2 

1 

1 

5 

Unnecessarily  Close  to  traffic 

(2) 

2 

2 

1 

2 

2 

1 

1 

2 

13 

Failing  to  communicate  to  important  traffic 

(2) 

4 

8 

2 

6 

5 

2 

10 

9 

5 

51 

c 

Failing  to  sound  fog  signal 

(1) 

2 

3 

5 

Failing  to  comply  with  the  rules  of  the  road 

(3) 

1 

2 

1 

4 

Poor  choice  of  maneuver,  where  the  rules  of  the 

(2) 

1 

1 

2 

4 

road  are  not  broken 

Operating  at  an  excessive  speed 

(2) 

2 

1 

1 

4 

Using  radar  on  improper  scale,  or  other  poor 

(1) 

5 

2 

4 

1 

12 

monitoring  condition 

Failing  to  detect  a  target 

(2) 

3 

2 

5 

Late  detection  of  a  target 

(1) 

1 

3 

4 

Not  monitoring  proper  channels 

(1) 

1 

1 

total  #  of  errors 

9 

13 

4 

25 

9 

7 

20 

13 

8 

108 

BRIDGE  MANAGEMENT 

ERRORS: 

Missing  a  few  Log  Book  entries,  less  than  40% 

(1) 

3 

3 

4 

4 

10 

6 

5 

7 

3 

45 

Missing  many  log  book  entries,  40%  or  greater 

(2) 

2 

5 

1 

2 

2 

4 

2 

2 

3 

23 

Failing  to  test  the  steering  gear 

(4) 

1 

1 

Failing  to  test  the  EOT 

(2) 

1 

1 

2 

Failing  to  Call  Engine  Room  prior  to  Arrival, 

(1) 

1 

1 

2 

and  at  Arrival 

Failing  to  set  radios  to  proper  channels 

(2) 

2 

1 

3 

Failing  to  notify  important  personnel 

(1) 

1 

i 

1 

1 

2 

Failing  to  prepare  anchor  before  entering 

(3) 

1 

1 

1 

restricted  waterway 

Failing  to  check  in  with  VTS  at  designated 

(1) 

1 

1 

1 

2 

4 

2 

2 

2 

15 

times  and  places 

total  #  of  errors 

5 

10 

6 

10 

15 

14 

10 

14 

11 

95 
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error  was  different.  The  other  type  of  error  which  occurred  relatively  frequently  was 
operating  on  the  wrong  side  of  the  channel  when  unnecessary.  As  the  table  shows,  this 
error  occurred  mostly  in  Scenarios  1  and  3,  and  while  it  might  be  understandable  for 
Scenario  3,  it  is  not  clear  why  this  should  have  happened  in  Scenario  1,  when  automatic 
updating  of  position  was  available. 

The  most  common  type  of  collision  avoidance  error  by  far  was  the  failure  to 
communicate  with  important  traffic.  While  this  probably  has  little  to  do  with  the 
presence  or  absence  of  ECDIS,  it  does  warrant  further  investigation  for  general  safety 
concerns. 

For  analysis,  the  errors  were  tabulated  and  weighted  based  on  the  nature  of  the  error,  with 
a  larger  penalty  given  to  errors  impacting  on  vessel  safety.  Each  mariner  received  an 
overall  scenario  score  as  well  as  scores  for  errors  in  navigation,  collision  avoidance  and 
bridge  management  events.  The  only  significant  correlation  obtained  was  between  the 
mariner's  reported  navigation  workload  (discussed  below)  and  the  navigation  error  score 
(Correl.  =.  33,  p  <  0.05),  indicating  that  navigation  errors  increased  as  navigation 
workload  increased. 


D.4  MEASURES  OF  FEATURE  AND  FUNCTION  USE 

D.4.1  Observer's  Tally 

Data  collected  for  ECDIS  feature/function  use  was  used  to  determine  what  features  are 
actually  used  under  various  circumstances.  Templates  that  were  replicas  of  the  control 
panels  of  each  ECDIS  unit  were  constructed  and  are  presented  in  Figures  D-8  and  D-9. 
During  the  experimental  nms,  one  experimenter  was  located  at  the  Human  Factors 
Station  and  monitored  the  ECDIS,  via  a  camera  trained  on  the  control  panel,  and  kept  a 
tally  of  all  the  functions  used.  A  tick  mark  were  placed  next  to  the  appropriate  feature, 
including  changes  in  chart  scales  and  alarm  warnings,  and  a  running  total  was  kept. 
These  were  analyzed  to  determine  the  importance  and  effectiveness  of  each 
feature/function. 
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SUBJECT _  OSL  FREQUENCY  COUNT  FORM 

SCENARIO! _ _  _  _ 
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Figure  D-8.  Template  of  OSL  Control  Panel  Used  to  Tally  Feature  Use 


Bearing  Line 


LAYOUTS  RADAR 


Figure  D-8.  Template  of  OSL  Control  Panel  Used  to  Tally  Feature  Use  (cont.) 


ROBERTSON  FEATURE/FUNCTION  TALLY  SHEET 


Figure  D-9.  Template  of  Robertson  Control  Panel  Used  to  Tally  Feature  Use 


D.4.2  Log  of  Paper  Chart  Use 


As  previously  mentioned,  all  mariners  were  provided  with  charts  covering  the  area  to  be 
transited.  These  were  kept  on  the  bridge  chart  table,  but  folded  so  their  use  would  be 
conspicuous  to  observers  at  the  Human  Factors  Station.  During  the  ECDIS-assisted 
transits,  mariners  were  provided  with  a  log  sheet  for  paper  chart  use,  in  which  they  were 
instructed  to  keep  track  of  when  they  referred  to  the  chart.  Included  in  the  log  was 
information  pertaining  to  the  time  they  referred  to  the  chart,  chart  number,  the 
information  obtained  and  reason  for  chart  use.  These  data  were  collected  and  analyzed  to 
identify  any  problems  encountered  with  ECDIS  and  what  additional  information  was 
required,  and  are  presented  in  Appendix  E. 

D.4.3  Post- Scenario  Questionnaires 

Detailed  post-scenario  questionnaires  were  developed  to  elicit  information  from  the 
mariners  regarding  a  range  of  issues  relating  to  their  most  recently  completed  transit. 

Each  questionnaire  was  scenario  specific,  and  included  mariner  self-ratings  of 
performance,  their  difficulty  in  using  and  confidence  in  the  ECDIS,  what  navigational 
and  charted  features  they  actually  used,  and  what  they  wanted  to  use.  In  addition,  there 
were  open-ended  questions  regarding  the  relative  safety  of  manually  updating  Ownship's 
position  on  the  ECDIS,  the  safety  and  usefulness  of  the  radar  overlay  and  ARP  A  targets, 
and  whether  ECDIS  supported  all  the  navigational  functions  they  wanted  to  perform. 
Figure  D-10  presents  a  composite  of  the  questions  for  each  scenario,  along  with  response 
data  where  appropriate. 

D.4.4  Post-experiment  Questionnaire 

Following  the  completion  of  all  nine  simulator  runs,  the  mariner's  filled  out  a  post¬ 
experiment  questionnaire,  which  was  detailed  and  global  in  scope,  asking  the  mariners  for 
their  expert  opinions  and  views  on  a  wide  range  of  navigational  issues  and  challenges.  In 
addition,  this  time  was  used  by  the  experimenters  to  re-visit  any  of  the  test  scenarios  with 
the  mariners  to  insure  that  they  had  proceeded  as  planned  and  that  no  simulator-related 
problems  had  interfered  with  their  performances.  Issues  covered  included  the  training 
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program,  bridge  configuration,  ECDIS  features  most  and  least  important,  the  ECDIS 
display,  radar  features,  safety  and  workload,  and  use  of  ECDIS  in  the  real  world  .  A 
sample  of  this  questionnaire  is  provided  as  Figure  D-1 1.  Note  that  responses  to  some  of 
the  questions  are  discussed  throughout  the  main  report. 
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MARINER# _  RUN# _ 

COMPOSITE  OF  POST-SCENARIO  QUESTIONNAIRES 

AND  RESPONSES 

Instructions:  PLEASE  ANSWER  THE  FOLLOWING  QUESTIONS 

REGARDING  YOUR  LAST  TRANSIT. 

Questions  common  to  all  scenarios: 

1 .  Rate  your  overall  navigational  performance  during  your  last  transit. 

1  2  3  4  5 


poor _ excellent 


Sc  1 

Sc  2 

Sc  3 

Sc  4 

Sc  5 

Sc  6 

Sc  7 

Sc  8 

Sc  9 

ave 

3.33 

4.20 

2.67 

4.08 

3.92 

3.42 

3.75 

3.50 

4.00 

sd 

0.58 

0.45 

0.58 

0.20 

0.49 

1.28 

0.61 

0.84 

0.00 

(n) 

_J2L_ 

(5) 

—0 

(6) 

0  . 

. 0 . 

(4) 

2.  Rate  your  overall  safety/collision  avoidance  performance  during  your  last  transit. 
1  2  3  4  5 


poor  _ _  excellent 


wm 

Ijgngi 

Gay 

KfSKrl 

■2iS9 

10^9 

mjKm 

^0^9 

m 

■n 

9f(b9 

miam 

999 

3 .  Rate  your  overall  management  of  the  bridge  during  your  last  transit 
1  2  3  4  5 


poor _ excellent 


Sc  1 

Sc  2 

Sc  3 

Sc  4 

Sc  5 

Sc  6 

Sc  7 

Sc  8 

Sc  9 

ave 

2.33 

2.60 

2.50 

3.67 

2.67 

3.25 

3.33 

3.08 

2.75 

sd 

0.58 

1.14 

1.32 

0.52 

0.52 

0.76 

0.52 

0.66 

0.50 

— 0 

m&m 

rnESm 

(3) 

(6) 

(6) 

(6) 

(6) 

msm 

(4) 

4.  Using  the  choices  given  below,  indicate  your  primary  and  secondary  method  of 
navigation  during  each  portion  of  the  transit,  (data  discussed  in  Section  5) 


Choices: 

1 .  Plotting  on  paper  chart 

2.  Conning  using  radar/ARPA 

3.  Conning  using  visual  means 

4 .  Conning  using  ECDIS  (choice  not  available  for  Sc  #  8  and  9) 

5.  Other  (please  specify) 


TRANSIT  SEGMENTS 
(Scenario  1) 

PRIMARY  METHOD 

SECONDARY 
METHOD  (if  any) 

Start  to  Ambrose  Light 

Ambrose  Light  to  Sea  buoy 

Sea  buoy  to  Ambrose  Channel 

Ambrose  Ch  to  Verraz.  Narrows  Br 

Verrazano  Narrows  Br  to  Anch 

Figure  D- 10.  Annotated  Composite  Post-Scenario  Questionnaire 
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TRANSIT  SEGMENTS 
(Scenario  2) 


BA  buoy  to  Ambrose  Light 


Ambrose  Light  to  Sea  buo 


Sea  buoy  to  Ambrose  Channel 


Ambrose  Ch  to  Verrazano  Narrows  Br 


Verraz.  Narrows  Br  to  Anchorage 


TRANSIT  SEGMENTS 

(Scenario  3) _ 


Anchorage  to  Verrazano  Narrows  Bndge 


Verraz.  Narrows  Br  to  Ambrose  Channel 


Ambrose  Channel  to  Sea  buo 


Sea  buoy  to  Ambrose  Light  _ 


Ambrose  Light  to  End 


TRANSIT  SEGMENTS 
(Scenario  4) 


Start  to  Ambrose  Light 


Ambrose  Light  to  Sea  buo 


TRANSIT  SEGMENTS 
(Scenario  5) 


SF  buoy  to  SF  Channel _ _ 


SF  Channel  to  Golden  Gate  Bndge 


Golden  Gate  Bridge  to  Alcatraz 


Alcatraz  to  Oakland  Bay  Bndge 


Oakland  Bay  Bridge  to  Anchorage 


TRANSIT  SEGMENTS 
(Scenario  6) 


Start  to  SF  buoy 


SF  buoy  to  SF  Channel 


SF  Channel  to  Golden  Gate  Bndge 


TRANSIT  SEGMENTS 
(Scenario  7) 


Anchorage  to  Oakland  Bay  Bridge 


Oakland  Bay  Bridge  to  Alcatraz 


Alcatraz  to  Golden  Gate  Bndge _ 


Golden  Gate  Bridge  to  SF  Channel 


SF  Channel  to  SF  buo 


SF  buoy  to  End 


TRANSIT  SEGMENTS 
(Scenario  8) 


Start  to  NA  buo 


NA  buoy  to  Ambrose  Light 


Ambrose  Light  to  Sea  buo 


Sea  buoy  to  Ambrose  Channel 


Ambrose  Ch  to  Verraz.  Narrows  Br 


Verrazano  Narrows  Br  to  Anch 


TRANSIT  SEGMENTS 
(Scenario  9) 


SF  buoy  to  SF  Channel  _ 


SF  Channel  to  Golden  Gate  Bridge 


Golden  Gate  Bridge  to  Alcatraz 


Alcatraz  to  Oakland  Bay  Bndge _ 


Oakland  Bay  Bridge  to  Anchorage 


Questions  specific  to  ECDIS  scenarios:  Sc  #  1.  2.  3.  4.  5.  6yand  JZ 

1 .  How  confident  were  you  in  using  ECDIS  as  your  primary  method  of  navigation? 

1  2  3  4  5 


not  confident _ very  confident 


Sc  1 

Sc  2 

Sc  3 

Sc  4 

Sc  5 

Sc  6 

Sc  7 

ave 

4.83 

4.80 

1.33 

4.50 

4.00 

1.17 

3.50 

sd 

0.29 

0.45 

0.58 

0.84 

1.10 

0.41 

1.64 

(») 

(?) . 

(6) 

(6)- 

(6) 

What  circumstances  lead  you  to  switch  to  another  method  (if  you  did)? 

Given  the  option,  would  you  have  preferred  to  navigate  your  last  transit 
using  the  ECDIS  or  a  paper  chart?  Please  explain. 

Figure  D-10.  Annotated  Composite  Post-Scenario  Questionnaire  (cont) 
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2.  Rate  the  overall  difficulty  of  operating  the  ECDIS. 

1  2  3  4  5 


difficult _ _ _  easy 


"STT 

Sc  2 

Sc  3 

Sc  4 

"sn" 

Sc  6 

Sc  7 

ave 

3.67 

3.40 

2.83 

4.00 

3.58 

2.00 

3.83 

sd 

1.53 

1.14 

1.89 

0.89 

0.49 

0.00 

0.75 

-JdL. 

(3) 

(5) 

(3) 

(6) 

-161_ 

(6) 

(6) 

If  you  experienced  any  problems  operating  the  ECDIS,  what  were  they? 

3 .  What  charted  features  would  you  have  liked  to  have  used  but  were  not  available  on 

the  ECDIS?  Did  you  refer  to  the  paper  chart  for  the  information?  (see  Section  6  and  Appendix 
E) 

4.  Did  the  ECDIS  support  you  in  all  of  the  navigational  functions  (e.g.,  calculating  ETA, 
taking  ranges/bearings,  etc.)  you  wanted  to  perform?  If  no,  what  functions  were  you 
unable  to  perform  and  why?  (see  Section  6  and  Appendix  E) 

5 .  Can  you  think  of  any  other  aspects  of  EdDIS  or  ECDIS-based  navigation  that  may 
potentially  lead  to  an  ambiguous  or  dangerous  situation  in  the  real  world? 

Questions  specific  to  OSL  scenarios:  Sc  #  1  2  rn=5i.  and  3  (n=3> 


Check 

total 

count 

the  navigational(text  box)data  you  actually  used  during  your  last  transit: 

total 

count 

(1) 

1.  Vessel  Lat/Long 

(8) 

6.  X-DS  -  Cross  Track  Distance 

(7) 

2.  Gyro 

(10) 

7.  CTS  -  Course  to  Steer 

(0) 

3.  Cursor  Lat/Long 

(6) 

8.  Radar  Overlay 

(11) 

(4) 

4.  Cursor  Range  and  Bearing 

5.  Target  Range  and  Bearing 

(  ) 

9.  OTHER  (please  list) 

What  navigational  data  would  you  have  liked  to  have  used  but  were  not  available  on  the 
ECDIS?  Did  you  get  the  data  from  another  source?  (data  presented  in  Section  6) 


2. 


3. 


Would  you  have  used  a  head  up  display  if  it  were  available  on  the  ECDIS?  Why  or 

why  not?  (Sc  #  3  only)  (Total  of  3  responses,  all  preferred  North  Up) 

How  important  is  it  to  have  the  option  of  both  north  up  and  head  up  displays 
available  on  the  ECDIS? 

1  2  3  4  5  ave  2.00 

not  important  very  important  sd  1.73 

(n=3) 


How  useful  was  having  depth  contours  displayed  on  the  ECDIS? 
1  2  3  4  5 


not  useful 


very  useful 


Sc  1 

Sc  2 

Sc  3 

ave 

3.00 

2.40 

1.50 

sd 

1.00 

0.55 

0.71 

M— 

_J[3} _ 

_ 

J21 _ 

Figure  D- 10.  Annotated  Composite  Post-Scenario  (Questionnaire  (cont.) 
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1 .  Check  the  navigational(text  box)data  you  actually  used  during  your  last  transit: 


total 

count 

(18) 

1. 

Crs  -  Course  (Gyro  Course) 

total 

count 

(10) 

7.  ARPAdata 

(17) 

2. 

Spd  -  Speed 

(20) 

8.  Range/Bearing 

(10) 

3. 

Cmg  -  Course  Made  Good 

(16) 

9.  Distance  to  next  waypoint 

(11) 

4. 

Smg  -  Speed  Made  Good 

(18) 

10.  ETA  to  next  waypoint 

(6) 

5. 

Set  -  Drift  Course 

(16) 

11.  Ship  speed/heading  vector 

(5) 

6. 

Drift  -  Drift  Speed 

(12) 

12.  Track  time^istory 

(  ) 

13.  OTHER  (please  list) 

What  navigational  data  would  you  have  liked  to  have  used  but  were  not  available  on  the 
ECDIS?  Did  you  get  the  data  from  another  source?  (see  Section  6  and  Appendix  E) 


2.  Check  the  charted  features  you  actually  used  during  your  last  transit: 

total  total 

count  count  i  •  x 

(16)  1.  Beacons  (15)  5.  Depth  markings  (soundings,  contour  lines) 

(21)  2.  Buoys  (8)  6.  Geographic  names 

(9)  3.  Dangers  (8)  7.  Lal/LongGrid 

(23)  4.  Traffic  Lanes  (7)  8.  Nav  Aid  Characteristics 

O  9.  OTHER  (please  specify) 


o  manual  update  scenari 


Were  you  able  to  manually  fix  your  position  on  the  ECDIS  to  your  satisfaction?  If 
not,  what  problems  did  you  encounter?  (see  Section  5.4.4) 

When  manually  fixing  your  position  on  the  ECDIS,  was  it  easier  or  more  difficult  as 
compared  to  tr^ition^  methods  using  the  paper  chart: 

Using  radar  overlay:  (Sc  #  3  only) 

1  2  3  4  5  ave  2j 

more  difficult  same  much  easier  sd  1j 


Using  ranges/bearings  and/or  lat  and  long:  (Sc  #  3  and  6) 
1  2  3  4  5 

more  difficult  same  much  easier 


ave  2.67 

sd  1.37 


What  in  particular  made  it  easier  or  more  difficult? 

3 .  Rate  the  level  of  navigational  safety  during  the  portion  of  the  transit  in  which  you  were 

manually  fixing  your  position  on  the  ECDIS,  as  compared  to  traditional  methods  using  the 


paper  chart. 

1  2 

3 

4 

5 

ave 

OSL 

2.00 

Robertson 

2.17 

less  safe 

same 

safer 

sd 

1.00 

0.75 

What  in  particular  do  you  think  made  manually  fixing  your  position  on  the  ECDIS  less  or 
more  safe? 


Figure  D-10.  Annotated  Composite  Post-Scenario  Questionnaire  (cont.) 


Questions  specific  to  radar  -  Sc  #  1  and  3.  and  ARPA  target  -  Sc  # 
4  overlay  scenarios 


1 .  Please  rate  the  usefulness  of  a  full  radar  overlay  (Sc  #  l  and  3;  rate  the  usefulness  of  overlaying 
selected  ARPA  targets ...  Sc  #  4)  for  performing  the  following  functions.  (Assume  that  the  radar 
overlay  display  is  as  good  as  a  stand  alone  display.) 

As  a  check  on  accuracy  of  primary  positioning  system: 

1  2  3  4  5 

not  useful  very  useful 

As  the  primary  method  for  positioning  vessei/chart: 

1  2  3  4  5 

not  useful  very  useful 

For  collision  avoidance/ARPA  tasks: 

1  2  3  4  5 

not  useful  very  useful 

2 .  Please  rate  the  safety  of  using  a  full  radar  overlay  (Sc  #  i  and  3;  rate  the  usefulness  of  overlaying 
selected  ARPA  targets ...  Sc  #  4)  when  performing  the  following  functions.  (Assume  that  the  radar 
overlay  display  is  as  good  as  a  stand  alone  display) 

As  a  check  on  accuracy  of  primary  positioning  system: 

1  2  3  4  5 

unsafe  practice  safe  practice 

As  the  primary  method  for  positioning  vessel/chart: 

1  2  3  4  5 

unsafe  practice  safe  practice 

For  collision  avoidance/ARPA  tasks: 

1  2  3  4  5 

unsafe  practice  safe  practice 

3  Do  you  have  any  safety  or  other  concems/observations  regarding  the  full  radar  overlay  (Sc  # 
1  and  3;  regarding  overlaying  selected  ARPA  targets  ...  Sc  #  4)?  If  yCS,  please  explain.  (Data  discussed  in 
Section  7) 


Figure  D-10.  Annotated  Composite  Post-Scenario  Questionnaire  (cont.) 
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ECDIS  PHASE  II 

POST-EXPERIMENT  DEBRIEFING  FORM 

THE  TEST  PROGRAM  AND  ITS  EFFECTS  ON  PERFORMANCE 


Test  Program 


MARINER# 


1 .  Rate  the  adequacy  of  the  training  you  received  in  using  the  ECDIS  devices: 

OSL  ECDIS:  ROPERTSON  ECPISi 

1  2  3  4  5  1  2  3  4  5 

poor  excellent  poor  excellent 

Comment  on  the  strengths  and  weaknesses  of  that  training  for  the  excercises  that  followed. 

Describe  any  aspects  of  either  ECDIS  on  which  you  would  like  to  have  been  better  trained/informed 
before  navigating  the  MA^  ECDIS. 

How  many  "watches"  did  you  complete  before  you  felt  comfortable  navigating  with  the: 

OSL  ECDIS:  RORERTSON  ECMSi  , 

(  )  1  (  )  2  (  )  more  than  2  (  )  never  (  )  1  (  )  2  (  )  more  than  2  (  )  never 


What  did  you  learn  from  using  the  devices  that  you  did  not  learn  fix)m  the  training? 

2 .  Rate  the  overall  realism  of  the  simulated  watches  you  participated  in: 

1  2  3  4  5 

poor  excellent 

Explain  what  you  felt  was  unrealistic. 

3 .  Were  there  any  aspects  of  the  simulator  or  simulations  that  may  have  caused  you  to  perform  in  a 
manner  significantly  different  from  how  you  would  perform  in  the  real  world? 

( )  Yes  ()  No  If  Yes,  please  explain: 

4.  Did  the  layout  of  the  bridge  influence  the  performance,  safety,  and  workload  during  the  runs? 

How  would  you  like  to  have  seen  the  bridge  arranged?  (Draw  a  picture  on  the  back  of  this  page  if  you 
wish.) 

5 .  Was  the  workload  appropriate  for  one  watchstander?  Where  there  differences  among  conditions  in  the 
appropriateness  of  workload?  How  would  you  have  preferred  to  see  the  workload  distributed? 

6.  Under  what  additional  circumstances,  conditions,  or  geographical  areas  would  you  have  liked  to  have 
the  opportunity  to  evaluate  ECDIS?  Why? 


The  following  questions  ask  for  your  professional  opinion  and  assessment  of  a  wide  range  of  EClDIS-related 
issues.  Please  try  to  visualize  this  technology  on  board  your  vessel  in  your  present  trade.  Assume  that  the 
technology  has  matured  and  technical  problems  relating  to  system  reliability,  system  bugs  and  the  like  have 
been  resolved.  Also  answer  the  questions  only  in  relation  to  the  navigation  monitoring  task.  Route 
planning  will  be  considered  on  another  questionnaire. 

Figure  D- 1 1 .  Post-Experiment  Questionnaire 
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MARINER# 


ENC 

7.  For  safe  and  efficient  ECDIS -based  position  monitoring,  indicate  which  charted  features  you  think 
should  ALWAYS  be  displayed  as  appropriate  to  the  chart  scale,  which  should  be  displayed  at  the 
USERS  OPTION  (i.e.,  the  user  can  turn  on/off  with  keystroke),  and  which  NEED  NOT  be 
displayed  at  all  on  the  electronic  chart.  Use  the  space  provided  for  comments,  and  list  any  additional 
charted  features  at  the  bottom. 


KEY:  A-  ALWAYS  DISPLAY  U-  USER  "ON/OFF"  OPTION  N-  NEED  NOT  DISPLAY 


CHARTED  FEATURES 

(A-U-N) 

COMMENTS 

LANDMASS  RELATED 

coastline/landmass 

1— iNnim  iiiiiiiii 

land  feature/characteristics  features 

names-landmasses,  islands,  points  etc. 

NAVIGATION  AID  RELATED 

indication  of  fixed  aids  to  navigation 

indication  of  floating  aids  to  navigation 

light  /  sound  characteristics 

physical  classification  (can/nun) 

physical  description  (e.g.  white  tower) 

visual  and  radar  conspicuous  features 

radio  characteristics  (RACON) 

AREA  DESIGNATIONS 

Federal  channel  lines 

navigation  lanes/fairways. 

prohibited  and  restricted  areas 

anchorages 

pilot  areas 

SEABED/BOTTOM  FEATURES 

spot  soundings 

bottom  contours 

bottom  characteristics 

cable/pipeline  areas 

Other  information 

indication  of  isolated  dangers  (wrecks, 
rocks,  overhead  bridges,  etc.) 

details  of  isolated  dangers 

indication  of  cautionary  notes 

details  of  cautionary  notes 

Lat/Long  grid  lines 

compass  rose 

magnetic  variation 

indication  of  units  of  depths  and  heights 

ENC  edition  date 

Star  the  three  charted  features  that  you  feel  are  most  important  to  the  safety  of  ECDIS  use.  When  and  why 
did  you  use  each  (or  would  you  have  liked  to  use  if  available).  Please  comment. 

Cross  out  the  three  charted  features  that  you  feel  are  least  important  to  the  safety  of  ECDIS  use. 

Figure  D- 1 1 .  Post-Experiment  Questionnaire  (cont.) 
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MARINER  # _  * 

8 .  Should  the  user  be  able  to  turn  on  and  shut  off  various  layers  of  attributes  (e.g.,  the  buoys,  text, 
beacons)?  Please  describe  any  situations  which  might  apply  to  your  answer. 

Is  chart-based  text  (e.  g.,  landmass  names  names,  navigation  aid  characteristics,  course)  useful  and/or 
necessary  on  the  electronic  chart?  Would  you  prefer  to  point  to  an  item  with  the  cursor,  for  example, 
and  get  information  on  a  separate  text  display? 


Monitoring 

9 .  What  capabilities  should  the  ECDIS  have  to  allow  the  operator  to  conduct  the  sames  navigational 
routines  and  plotting  as  done  on  paper  chart  with  a  ruler? 

(  )  triangles,  dividers,  pencil,  etc 
(  )  variable  range  marker 
(  )  fixed  range  marker 

(  )  electronic  bearing  line  (fixed  and  free)  with  built  in  VRM 
(  )  navlines 

(  )  any  others?  (please  list) 


Figure  D-1 1. 


Post-Experiment  Questionnaire  (cont.) 
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MARINER  # -  Monitoring 

10.  Indicate  which  ECDIS-generated  features  and  functions  you  think  should  ALWAYS  be 

displayed  as  appropriate  to  the  chart  scale,  which  should  be  displayed  at  the  USERS  OPTION  (i.e., 
the  user  can  turn  on/off  with  keystroke),  and  which  NEED  NOT  be  displayed  at  all.  Use  the  space 
provided  for  comments,  and  list  any  aditional  ECDIS-generated  features  and  functions  at  the  bottom. 


KEY:  A  -  ALWAYS  DISPLAY  U-  USER  "ON/OFF"  OPTION  N  -  NEED  NOT  DISPLAY 


FEATURES  &  FUNCTIONS 

(A-U-N) 

COMMENTS 

OWNSHIP  INFORMATION 

ownship  outline 

Past  track 

vector  of  ownship  heading  and  speed 

vector  of  course  and  speed  made  good 

NAVIGATION  TOQlS  AND 

DATA 

display  planned  trackline 

course  to  steer  (trackline) 

display  waypoint  and  waypoint  number 

display  wheel  over  points/tum  radius 

display  range  rings 

provide  method  for  manual  fix  taking 

display  fix  marker  and  time 

display  dead  reckoned  position  and  time 

CALCULATED/SPECIAL 

display  ownship's  safety  depth  contour 

display  current  vectors 

display  visual  limits  of  lights 

display  selected  ARPA  targets 

display  overlay  of  actual  radar 

display  chart  north  up  and  course  up 

Set  and  Drift 

ETA  to  waypoint 

zoom  in/  out  function 

navigation  fault  alarm  (e.g.  GPS  down) 

off  track  alarm 

grounding  alarm 

OTHER  INFORMATION 

scale  bar 

chart  scale  boundaries 

Star  the  three  ECDIS-generated  features  and  functions  that  you  feel  are  most  important  to  the  safety  of  ECDIS 
use.  When  and  why  did  you  use  each  (or  would  you  have  liked  to  use  if  available).  Please  comment. 


Cross  out  the  three  ECDIS-generated  features  and  functions  that  you  feel  are  least  important  to  the  safety  of 
ECDIS  use. 


Figure  D- 1 1 .  Post-Experiment  Questionnaire  (cont.) 
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MARINER  # _  DisBlay 


1 1 .  How  important  is  it  to  have  the  option  of  both  true  and  relative  chart  motions  available? 

1  2  3  4  5 

not  important  v«y  important 

What  advantages  or  problems  do  you  see  in  having  both  available?  (Please  consider  both  experienced 
and  more  junior  officers.) 

12.  How  important  is  it  to  have  the  option  of  both  north  up  and  head  up  displays  available? 

1  2  3  4  5 

not  important  very  important 

What  advantages  or  problems  do  you  see  in  having  both  available?  (Please  consider  both  experienced 
and  more  junior  officers.) 

1 3 .  How  important  is  it  to  be  able  to  offset  ownship  from  the  center  of  the  screen  as  did  the  Robertson 
device?  Is  the  offset  important  if  you  have  more  than  one  window  as  did  the  OSL? 

14.  How  many  displays  do  you  feel  are  necessary  to  fulfill  the  independent  ECDIS  functions  of  route 
planning,  navigation  monitoring,  and  display  of  auxiliary  information?  Is  it  sufficient  to  switch  back  and 
forth  as  on  the  Robertson  ECDIS,  or  would  you  prefer  that  the  navigation  monitoring  display  is  always 
visible  as  on  the  OSL? 

Radar 

1 5 .  Do  you  think  radar/ARPA  features  should  be  integrated  into  the  ECDIS?  What  are  the  advantages  and 
disadvantages  of  such  integration?  What  would  be  the  effects  on  safety  and  on  workload? 

Should  that  integration  be  as  a  complete  overlay,  like  the  OSL  or  as  A^A  targets  alone,  like  the 
Robertson?  What  are  the  advantages  or  disadvantages  of  each  possibility?  What  would  be  the  effects  on 
safety  and  workload  of  each  possibility? 

Could  such  an  integrated  device  ever  replace  one  of  the  radars  on  the  bridge? 

General 

1 6.  What  do  you  feel  are  the  best  and  worst  aspects  of  ECDIS  as  presented  by  the  two  devices  you  saw? 
What  improvements  would  you  suggest? 

Safg  nav 

17.  Do  you  think  that  ECDIS  can  replace  the  paper  chart  in  all  operational  circumstances?  What  could  be 
added  or  improved  in  order  to  make  this  possible?  Are  there  any  aspects  of  ECDIS  technology  that  make 
it  an  unacceptable  replacement  for  the  paper  chart? 


Figure  D- 1 1 .  Post-Experiment  Questionnaire  (cont.) 
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MARINER# _ 

USE  OF  ECDIS  IN  THE  REAL  WORLD 

18.  How  would  you  expect  an  ECDIS  to  be  used  on  the  last  ship  on  which  you  sailed?  Where  would  it  be 
placed  on  the  bridge?  How  would  it  be  incorporated  into  watchstanding  practices?  Would  you  expect  it 
to  be  accepted? 

19.  Would  you  trust  ECDIS  devices  in  the  real  world?  Would  you  have  confidence  in  watchstanding  using 
such  devices?  What  factors  would  limit  your  confidence?  Are  they  primarily  device  characteristics, 
positioning  accuracy,  training,  or  others? 

20.  Do  you  see  a  place  for  ECDIS  in  piloting?  How  would  you  expect  the  wide-spread  availability  of  these 
devices  to  affect  piloting  practice  and  regulation? 

2 1 .  Does  an  integrated  navigation  display  --  chart  and  radar  —  have  a  function  in  reduced  visibility  beyond 
that  played  by  radar  alone?  How  would  you  expect  the  wide-spread  availablity  of  these  devices  to  affect 
practice  and  regulation? 

22.  Does  the  present  training  and  certification  for  watchstanding  using  paper  charts  and  radar  prepare  for 
ECDIS  use?  If  not,  what  kinds  of  training,  certification,  etc.  do  you  think  would  be  necessary  for  the 
effective  use  of  these  devices?  In  the  long  run  do  you  see  ECDIS  as  increasing  or  decreasing  the  general 
level  of  skill  needed  by  a  watchstander? 

23 .  What  pitfalls  or  dangers  in  using  ECDIS  might  a  more  junior  officer  experience? 

24.  Was  the  wwkload  appropriate  for  one  watchstander?  How  would  you  have  preferred  to  see  the 
worklo^  distributed?  What  factors  would  affect  the  workload  and  its  distribution?  Do  you  see  the  need 
or  possibility  of  changing  staffing  standards  with  the  availability  of  the  ECDIS  technology? 

25 .  Does  the  use  of  the  ECDIS  devices  have  any  relation  to  fatigue?  Would  it  increase  or  decrease  the 
fatigue  caused  by  standing  a  watch?  For  a  fetigued  watchstander,  would  it  make  his  job  easier  or 
harder? 

26.  Design  your  dream  ECDIS.  First  consider  the  ECDIS  as  it  would  be  used  during  route  monitoring. 
Then  consider  passage  planning.  Mention  which  aspects  of  either  (or  neither)  ECDIS  you  worked  with 
that  you  would  include,  modify,  or  eliminate.  Try  to  consider  as  many  circumstances  and  conditions  as 
you  can  to  design  an  ECDIS  that  will  contribute  to  safer  navigation  and  lower  navigational  workload  for 
the  user.  Use  the  back  of  this  sheet  if  necessary.  Happy  dreams!! 


THANK  YOU  FOR  PARTICIPATING!! 


Figure  D- 1 1 .  Post-Exp)eriment  Questionnaire  (cont.) 
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APPENDIX  E.0 


SUPPLEMENTARY  MATERIAL  FOR 

SECTIONS  5.0, 6.0,  AND  7.0  RESULTS  AND  DISCUSSION 


E.  1  DISCUSSION  OF  STATISTICAL  ANALYSES 

Many  of  the  performance  scores  discussed  below  and  throughout  the  Main  report  were 
subjected  to  statistical  analyses.  In  particular,  the  Repeated  Measures  Analysis  of 
Variance  (ANOVA)  procedure  was  used,  and  Single  Degree  of  Freedom  Contrasts  were 
performed.  It  is  the  F  values  and  significance  levels  of  these  contrasts  that  are  cited  in  the 
text  of  the  Main  Report. 


Note  in  these  figures  that  in  addition  to  the  usual  p-value,  p-values  for  which  the  degrees 
of  freedom  are  adjusted  using  both  the  Greenhouse-Geisser  (G-G)  and  Huynh-Feldt  (H-F) 
epsilon  factors  are  also  provided.  These  epsilon  factors  take  into  account  any  violations 
in  the  assumptions  about  the  correlation  between  the  observations.  The  G-G  tends  to  be 
conservative,  while  the  H-F  tends  to  be  somewhat  liberal.  A  discussion  of  these  factors 
can  be  found  in  Maxwell  and  Delaney  (1990)  as  well  as  the  operations  manual  for 
Abacus  Concepts  SuperAnova  (1989).  As  a  general  rule,  the  most  conservative  of  the 
three  p-values  is  reported  in  the  text. 

Due  to  occasional  malfunctions  of  the  prototype  ECDIS  devices  or  to  other  problems, 
some  runs  were  lost  While  the  means  and  standard  deviations  reported  in  the  text  reflect 
only  data  actually  collected,  estimated  values  were  used  for  the  both  ANOVA  and 
supporting  contrasts. 

E.2  PRELIMINARY  ANALYSIS  OF  THE  NASA  TASK  LOAD  INDEX 

The  NAS A-TLX  subjective  rating  scales  were  used  to  evaluate  the  mariner's  experience 
of  workload.  Appendix  D  contains  sample  materials  and  the  Main  Report  presents  the 
more  striking  findings.  This  section  presents  some  of  the  more  preliminary  findings  as 
well  as  supporting  tables  of  statistical  data. 

As  a  preliminary  examination  of  the  effects,  the  mean  (and  standard  deviation)  workload 
scores  for  each  scenario  are  shown  in  T able  E- 1  below. 
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Table  E-1.  Mean  Workload  Reported  for  Each  Scenario  on  a  Zero-  to  -100 
Workload  Scale 


Source  of 
Workload 

1 

2 

3 

Scenario 

4  5  6 

7 

8 

9 

Navigation 

mean 

33.84 

39.00 

73.42 

24.42 

38.33 

63.82 

40.69 

49.86 

58.17 

sd 

12.61 

16.72 

13.20 

6.12 

13.03 

17.95 

9.79 

13.53 

7.08 

n 

3 

5 

3 

6 

6 

6 

6 

6 

4 

Collision 

mean 

63.94 

64.25 

60.03 

57.36 

48.87 

51.31 

55.80 

52.78 

63.08 

Avoidance 

sd 

20.43 

12.21 

16.14 

17.35 

15.51 

10.45 

14.60 

18.37 

18.04 

n 

3 

5 

3 

6 

6 

6 

6 

6 

4 

Bridge 

mean 

54.60 

58.76 

67.54 

46.24 

57.53 

50.56 

46.75 

54.81 

54.77 

Management 

sd 

7.12 

18.06 

24.59 

15.24 

15.27 

16.78 

9.36 

12.04 

9.90 

n 

3 

5 

3 

6 

6 

6 

6 

6 

4 

An  underlying  assumption  made  when  analyzing  these  data  is  that  navigation  workload 
(as  opposed  to  collision  avoidance  or  bridge  management)  would  be  most  dramatically 
influenced  by  introducing  ECDIS  to  the  bridge.  Therefore,  although  some  analyses  were 
performed  for  the  collision  avoidance  and  bridge  management  data,  most  of  the  effort 
was  focused  on  navigation  workload,  including  statistical  comparisons  between  ECDIS 
scenarios  and  between  ECDIS  versus  baseline  scenarios.  As  Table  E-1  shows,  the 
assumption  is  supported  in  that  both  collision  avoidance  and  bridge  management 
workloads  were  fairly  consistent  across  the  scenarios,  while  navigation  workload  was 
much  more  variable.  This  table  shows  that  the  NASA-TLX  workload  measurement 
technique  was  indeed  sensitive  to  variations  in  navigation  workload  represented  by  the 
different  bridge  conditions  presented  to  the  mariners. 

Another  assumption  made  when  comparing  the  scenarios  on  any  of  the  measures,  was 
that  the  overall  distribution  of  workload  was  similar  across  the  scenarios  (via  the  mix  of 
navigation,  collision  avoidance  and  bridge  management  tasks).  To  verify  this 
assumption,  the  overall  workload  scores  for  each  of  the  three  bridge  tasks  was  weighted 
by  the  proportion  of  time  the  mariner  reported  spending  on  each  task .  A  single  score 
was  computed  for  each  scenario,  which  reflected  a  combination  of  workload  for  all  three 
tasks  and  the  percent  of  time  spent  on  each.  The  mean  overall  workload  for  each  scenario 
is  in  shown  in  Figure  E-1  below.  No  analyses  were  done  on  these  data. 
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E.3  SCENARIO  ANALYSES  OF  ECDIS  EFFECTS  ON  WORKLOAD 

The  effect  of  ECDIS  on  mariner  workload  was  described  in  Section  5  of  the  Main  Report. 
The  effect  was  further  analyzed  for  the  individual  scenarios.  Since  navigation-related 
activities  were  the  most  clearly  affected  by  the  presence  or  absence  of  ECDIS,  and  since 
several  of  the  mariners  indicated  that  they  consider  navigation  to  be  a  very  separate 
thought  process  from  collision  avoidance,  the  following  discussion  is  concentrated  on  the 
findings  for  navigation  workload.  While  there  were  some  statistically  significant 
differences  obtained  in  mean  collision  avoidance  workload  among  scenarios,  the  presence 
or  absence  of  ECDIS  did  not  produce  a  consistent  effect.  The  mean  level  of  workload  for 
bridge  management-related  tasks  does  not  appear  to  be  affected  by  the  introduction  of  the 
ECDIS  to  the  bridge. 

The  mean  navigation  workload  for  the  conventional  bridge  (Scenario  8  )  as  compared  to 
OSL  with  automatic  positioning  (Scenario  2)  is  shown  in  Figure  E-2.  The  reduction  in 
mean  navigation  workload  between  these  scenarios  is  attributable  only  to  the  availability 
of  ECDIS  because  the  routes  into  New  York  Harbor  were  identical  and  both  transits  were 
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Mean  Navigation 
Workload 


Bridge)  w/Auto  Bridge)  w/Auto 

Update)  Update) 


Figure  E-2.  Mean  navigation  workload  during  conventional  bridge  scenarios  (8  and  9) 
each  versus  a  comparable  ECDIS  scenario  (2  and  5) 


made  in  clear  visibility.  The  difference  in  mean  navigation  workload  between  these  two 
scenarios  was  statistically  significant  (F  =  3.96,  p  <  0.07).  A  second,  similar  comparison 
is  also  presented  in  Figure  E-2.  In  this  case,  navigation  workload  on  a  conventional 
bridge  (Scenario  9)  is  compared  with  the  Robertson  ECDIS  (Scenario  5),  with  both 
transits  going  from  the  coastal  zone  into  San  Francisco  Harbor  in  clear  visibility.  Again, 
there  is  a  relatively  large  reduction  in  navigation  workload  that  is  statistically  significant 
(F  =  8.84,  p  <  0.03),  further  supporting  the  hypothesis  that  ECDIS  does  reduce  perceived 
navigational  workload. 

The  supporting  ANOVA  table  and  contrasts  for  the  workload  scores  discussed  above  and 
in  the  Main  Report  are  presented  in  Figure  E-3.  Figure  E-4  presents  similar  tables  and 
contrasts  for  the  analysis  of  percent  of  time  spent  on  navigation,  collision  avoidance  and 
bridge  management  tasks  discussed  in  the  Main  Report.  The  analyses  performed  on  the 
primary  method  of  navigation  data  are  given  in  Figure  E-5,  and  Figure  E-6  presents  the 
ANOVA  table  for  average  situational  awareness.  Various  correlations  between  these 
measures  can  be  found  in  Table  E-2.  The  charted  features  and  navigational  data 
mentioned  on  the  paper  chart  use  forms  and  in  the  questionnaires  are  compiled  and 
presented  in  Table  E-3.  Finally,  a  summary  of  the  average  cross-track  distance  data  and 
the  t-values  for  certain  comparisons  are  presented  in  Table  E-4,  followed  by  a  complete 
set  of  the  composite  trackplots  in  Figures  E-7  through  E-15. 


E-4 


Source 

df 

Sum  of  Squares 

Subject 

5 

5429.312 

Bridge  Task 

2 

2559.747 

Bridge  Task  *  Subject 

10 

6616.970 

Scenario 

8 

4457.336 

Scenario  *  Subject 

40 

5256.721 

Bridge  Task  *  Scenario 

16 

6570.921 

Bridge  Task  *  Scenario  *  Subj... 

80' 

7354.348 

Dependent:  Workload  Scores 

Table  of  Epsilon  Factors  for  df  Adjustment 
[}ependent:  Workload  Scores 


Bridge  Task 
Scenario 

Bridge  Task  *  Scenario 

NOTE:  Probabilities  are  not  corrected  for  values 
of  epsilon  greater  than  1 . 


G-G  Epsilon  H-F  Epsilon 


.933 

1.467 

.464 

1.972 

.204 

.633 

SINGLE  DEGREE  OF  FREEDOM  CONTRASTS 
Effect:  Bridge  Task  *  Scenario 
Dependent:  Navigation  Workioad  Scores 


ROB  ECDIS  vs.  Conventional  Bridge 


OSL  ECDIS  w/  Auto  Positioning 
vs  OSL  Manual  Positioning 


Cell  Weight  Cell  Weight 


NAV,  Sc5 

1.000 

NAV.  Scl  r 

1.000 

NAV,  Sc9 

-1  .000 

NAV,  Sc3  L 

-1 .000 

df  1  df  1 


Sum  of  Squares  813.124  Sum  of  Squares  1902.903 


Mean  Square  813.1  24 

Mean  Square  1902.903 

F-Value  8.845 

F-Value  20.700 

P-Value  .0039 

P-Value  .0001 

G-G  .0338 

G-G  .0079 

H-F  .0111 

H-F  .0003 

Figure  E-3.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 

Contrasts  for  Workload  Scores 
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SINGLE  DEGREE  OF  FREEDOM  CONTRASTS  (contt 
Effect:  Bridge  Task  *  Scenario 
Dependent:  Navigation  Workload  Scores 

ROB  ECDIS  w/  Auto  Positioning 

OSL  ECDIS  vs  Conventional  Bridge  vs  ROB  ManualPosJttojilnq 


Ceil  Weight  Cell  Weight 


NAV,  Sc2 

1.000 

NAV. 

Sc5 

1.000 

NAV,  Sc8 

-1,000 

NAV, 

Sc6 

-1.000 

df 

1 

df 

1 

Sum  of  Squares 

363.947 

Sum  of  Squares 

1948.965 

Mean  Square 

363.947 

Mean  Square 

1948.965 

F-Value 

3.959 

F-Value 

21.201 

P-Value 

.0500 

P-Value 

,0001 

G-G 

.0786 

G-G 

.0075 

H-F 

.0671 

H-F 

.0003 

ROB  ECDIS  Manual  Positioning 
vs  Conventional  Bridge 


OSL  ECDIS  Manual  Positioning 
vs  Conventional  Bridge 


Cell  Weight  Cell  Weight 


NAV,  Sc6 

1.000 

NAV.  Sc3 

1.000 

NAV,  Sc9 

-1 .000 

NAV,  Sc8 

-1 .000 

df 

1 

df 

1 

Sum  of  Squares 

244.352 

Sum  of  Squares 

530.630 

Mean  Square 

244.352 

Mean  Square 

530.630 

F-Value 

2.658 

F-Value 

5.772 

P-Value 

.1070 

P-Value 

.0186 

G-G 

.1059 

G-G 

.0555 

H-F 

.1180 

H-F 

.0330 

Both  ECDIS  w/  Auto  Positioning 
vs  Both  ECDIS  w/  Manual  Positioning 


NAV,  Sc1 
NAV,  Sc3 
NAV.  Sc5 
NAV,  Sc6 


df 

1 

m  of  Squares 

3851.731 

Mean  Square 

3851.731 

F-Value 

41.899 

P-Value 

.0001 

G-G 

.0012 

H-F 

.0001 

Cell  Weight 
.500 
-.500 
.500 
-500 


All  ECDIS  w/  Auto  Positioning  vs 
Conventional  Bridge 


NAV.  Sc1 
NAV.  Sc2 
NAV.  Sc4 
NAV,  Sc5 
NAV,  Sc7 
NAV,  Sc8 
NAV.  Sc9 


Cell  Weight 


df 

Sum  of  Squares 
Mean  Square 
F-Value 
P-Value 


G-G 

H-F 


1 

2270.093 

2270.093 

24.694 

.0001 

.0052 

,0001 


Figure  E-3.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 
Contrasts  for  Workload  Scores  (cont.) 
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Source  df  Sum  of  Squares  Mean  Square  F-Value  P-Value  GG  H-F 


Subject 

5 

.544 

.109 

Bridge  Task 

2 

15200.753 

7600.376 

9.394 

.0051 

.0114 

.0051 

Bridge  Task  *  S... 

1  0 

8090.714 

809.071 

Scenario 

8 

.826 

.103 

1.000 

.4512 

.3632 

.3632 

Scenario  *  Subj... 

40 

4.128 

.103 

Bridge  Task  *  S... 

1  6 

6077.193 

379.825 

3.281 

.0002 

.0540 

.0087 

Bridge  Task  *  S... 

80 

9260.174 

115.752 

Dependent:  Percent  Time 


Table  of  Epsilon  Factors  for  df  Adjustment 
Dependent:  Percent  Time 


Bridge  Task 
Scenario 

Bridge  Task  *  Scenario 

NOTE:  Probabilities  are  not  corrected  for  values 
of  epsilon  greater  than  1 . 

SINGLE  DEGREE  OF  FREEDOM  CONTRASTS 


G-G  Epsilon  H-F  Epsilon 


.760 

1.022 

.125 

.125 

.178 

.437 

All  ECDIS  w/  Auto  Positioning  All  ECDIS  w/  Auto  Positioning 

vs  Conventionai  Bridge  vs  Conventional  Prld.gg 

Effect:  Bridge  Task ‘Scenario  Effect:  Bridge  Task*  Scenario 

Dependent:  Percent  Time  on  Navigation  Dependent:  Percent  Time  on  Coilision 

Avoidance 


Cell  Weight 

Ceil  Weight 

NAV,  Sc1 

.200 

C/A,  Scl 

.200 

NAV,  Sc2 

.200 

C/A.  Sc2 

.200 

NAV,  Sc4 

.200 

C/A,  Sc4 

.200 

NAV.  Sc5 

.200 

C/A,  Sc5 

.200 

NAV.  Sc7 

.200 

C/A.  Sc7 

.200 

NAV.  Sc8 

-.500 

C/A,  Sc8 

-.500 

NAV.  Sc9 

-.500 

C/A,  Sc9 

-.500 

df  1  df  1 


Sum  of  Squares 

648.771 

Sum  of  Squares 

583.393 

Mean  Square 

648.771 

Mean  Square 

583.393 

F-Value 

5.605 

F-Value 

5.040 

P-Value 

.0203 

P-Value 

.0275 

G-G 

.0574 

G-G 

.0632 

H-F 

.0459 

H-F 

.0543 

Figure  E-4.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 
°  Contrasts  for  Percent  of  Time  Spent  on  Bridge  Tasks 
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Source  df  Sum  of  Squares  Mean  Square  F-Value  P-Value  G-G  H>F 
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Figure  E-5.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 
Contrasts  for  Primary  Method  of  Navigation 
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Figure  E-5.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 
Contrasts  for  Primary  Method  of  Navigation  (cont.) 


Source 

df 

Sum  of  Squares 

Mean  Square 

F-Value 

P-Value 

G-G 

H-F 

Subject 

4 

4.723 

1.181 

Scenario 

8 

2.045 

,256 

1.212 

.3237 

.3481 

.3266 

Scenario  *  Subject 

32 

6.751 

.211 

Dependent:  Average  Situational  Awareness 


Table  of  Epsilon  Factors  for  df  Adjustment 
Dependent:  Average  Situational  Awareness 


Scenario 


G-G  Epsilon  H-F  Epsilon 


.319 


.933 


Figure  E-6.  Analysis  of  Variance  (ANOVA)  Tables  and  Single  Degree  of  Freedom 
Contrasts  for  Average  Situational  Awareness 
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Table  E-2.  Correlation  Analyses  Between  Measures 


Source 

Correlation 

P- 

Value 

90% 

Lower 

90% 

Upper 

#of 

observ 

ations 

Navigation  workload  score  correlated 
with  expert  mariner-observer's  tally  of 
navigation  errors 

.330 

.0486 

.057 

.558 

36 

Navigation  workload  score  correlated 
with  proportion  of  time  spent  on 
collision  avoidance 

-.575 

.0002 

-.736 

-.352 

36 

Navigation  workload  score  correlated 
with  proportion  of  time  spent  on 
navigation 

.508 

.0013 

.267 

.689 

36 

Average  situational  awareness  rating 
correlated  with  total  number  of  chart  re¬ 
creation  drawing  errors 

-.126 

.1158 

-.253 

.006 

158 

Average  situational  awareness  rating 
correlated  with  total  number  of  chart  re¬ 
creation  drawing  errors  for  transit 
segments  w/  ecdis  used  as  primary 
method  of  navigation 

-.244 

.0370 

-.419 

-.053 

73 
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Table  E-3.  ECDIS  Features  and  Navigational  Data  Missed/W anted 


Charted  Features 

Navigational  Data 

Paper  Chart  Use 

Wanted  on  ECDIS 

Wanted  on  ECDIS 

Scenario  1 

Mariner 

confirm  buoy  #,  refer  to 

buoy/nav  aid  ID 

time/dist  info,  SOG,  CMG 

#2 

personal  notes  re:  call  E/R 

M#3 

better  navig.  mark 
presentation 

AREA  target  vectors 

M#4 

buoy  #  at  13  &  14,  17  &  18, 

buoy  #'s  &  symbols. 

course  change  alarm,  chart 

verify  targets  as  buoys, 

soundings  and  other  aids; 

change  alarm,  ETA  to  next 

sounding  on  courseline 

more  detail 

waypoint,  ARPA,  time  to  go 
to  cursor  range  &  bearing, 
target  vectors  &  #’s 

Scenario  2 

M#2 

check  buoy  #’s  and  other  info 

buoy  #’s 

ETA  info,  heading  vector, 
course/dist  b/w  waypoints 

M#3 

checked  name  of  Chapel  Hill 

text  naming  Chapel  Hill 

ETA  to  waypoint 

channel,  check  Ambrose 
tower  info 

channel 

M#4 

soundings,  buoy  #'s 

symbols 

M#  5 

next  course,  buoy  #'s, 

numerical  indication  of 

courses,  course  &  time  to 

anchorage  area 

depths  (assoc  w/  contours) 

waypoint,  ETA 

M#6 

distance  &  bearing  to 
Ambrose,  buoy  #'s,  buoy 
lights,  eyeballed  anchorage 
position 

buoy  info 

target  info,  ETA 

Scenario  3 

M#2 

check  course,  buoys,  bearing 

buoy/nav  aid  info  (#'s) 

set/drift  adjustment  when 

to  light  tower 

DRing 

M#3 

M#4 

info,  position,  buoys,  buoy 
#'s 

used  passage  plan  (?) 

courselines 

Scenario  4 

M 

check  buoy  name 

nav  aid  ID  data 

CMG,  SMG,  set,  drift 

M#5 

check  soundings,  buoy  #’s  in 

variable  vector  line  similar  to 

Ambrose 

Raytheon  ARPA 

M#6 

plot  position  (twice)  in 

CPA  &  TCPA  to  cursor. 

relation  to  Ambrose  to  check 
ETA,  speed,  &  dist.  to  go 

ETA  to  waypoint 

M#7 

check  on  aids  at  Ambrose, 

buoy  names/  #’s 

next  course  and  distance  as  a 

yellow  buoy  on  ECDIS  but 

way  of  monitoring  entire 

not  on  chart 

route 

Scenario  5 

M#  2 

buoy  &  geographic  points  ID 
data 

M#3 

check  parallel 

parallel  indexing  details 

M#5 

check  buoy  #,  sounding, 

bridge  info,  sounding, 

extending  vector  line 

check  bridge  span  C-D 

anchorage  location 
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Table  E-3.  ECDIS  Features  and  Navigational  Data  MissedAV anted  (cont.) 


Paper  Chart  Use 

Charted  Features 
Wanted  on  ECDIS 

Navigational  Data 
Wanted  on  ECDIS 

Scenario  5  (cont.) 

Marina- 

#6 

M#7 

ref.  name  Pt.  Bonita,  re.  C-D 
span  for  Bay  Br,  change 
charts,  water  depth 
check  bridge  pier  #’s, 
anchorage  limits 

bridge  info 

outline  of  anchorage  limits, 
buoy  names/#'s,  bridge  pier 
#'s  (letters) 

CPA  &  TCPA  to  cursor 

radar/ARPA  overlay 

Scenario  6 

M#2 

M#3 

M#4 

M#5 

M#6 

M#7 

check  position  (twice),  check 
parallel  lands  end 

plot  fix  at  #  5  (fe  6 

plot  position  (6x),  check  SF 
buoy,  change  chart 

buoy  names,  #'s,  point 
names 

ANTS  data  in  DR  mode 
radar  lock  onto  overlay,  a 
buoy  or  other  fixed  object 
position 

variable  VRM,  range  & 
bearing  (?) 

accurate  position 

Scenario  7  | 

M#2 

M#3 

M#4 

M#5 

M#6 

M#7 

check  D  tower,  position 
(twice),  position  vs  ECDIS, 
course  bar  channel,  SF  buoy 
span  info,  course  ref, 
waypoint  ref  (twice) 

course  depth,  wreck  &  bridge 
clearance,  swingwide, 
precautionary  area,  change 
chart,  MN  ship  channel,  plot 
position  (4x) 
buoy  #  for  course  change, 
buoy  name  (Hard.  Rk), 
colregs  demarcation  line, 
check  for  buoy  #8  (missing 
from  ECDIS) 

text  informaiton  on  buoys, 
etc. 

aid  #’s,  auto  update  to 
coastal/approach  charts 

buoy  names/#’s 

proper  CTS  info 

targets 

ETA  to  cursor 

radar/ARPA 
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Table  E-4.  Mean  Cross-Track  Distances  For  Scenario  Comparisons 


Location 

Mean 

Cross-Track  Distance 
of  Scenarios  Compared 

t- Value 

p- Value 

San  Francisco 

ScenS 

Scen9 

SF  channel  entrance 

322.69 

355.93 

-0.61 

0.58 

Golden  Gate  Bridge 

107.41 

385.94 

-3.07 

0.05 

Alcatraz 

58.83 

331.31 

-2.44 

0.09 

Alcatraz  2nd  turn 

96.62 

323.70 

-2.28 

0.06 

San  Francisco 

ScenS 

Scen6 

SF  channel  entrance 

322.69 

378.49 

-0.88 

0.41 

SF  channel  -  WP4 

301.62 

610.33 

-1.12 

0.31 

Golden  Gate  Bridge 

107.41 

155.51 

-0.38 

0.72 

San  Francisco 

Scen6 

Scen9 

SF  channel  entrance 

378.49 

355.93 

0.09 

0.93 

SF  channel  -  WP4 

610.33 

355.97 

-0.90 

0.43 

Golden  Gate  Bridge 

155.51 

385.94 

-1.86 

0.16 

New  York 

Scen2 

ScenS 

Sea  Buoy 

633.41 

1090.41 

-1.20 

0.31 

Ambrose  1st  turn 

112.67 

150.93 

-1.23 

0.30 

Ambrose  2nd  turn 

390.82 

324.97 

0.07 

0.95 

Verrazano  Bridge 

118.61 

190.76 

-1.71 

0.18 

New  York 

Scen3 

ScenS 

Sea  Buoy 

470.49 

1090.41 

4.48 

0.04 

Ambrose  1st  turn 

423.75 

150.93 

2.05 

0.17 

Ambrose  2nd  turn 

724.31 

324.97 

2.18 

0.16 

Verrazano  Bridge 

489.80 

190.76 

1.04 

0.40 

New  York 

Scen3 

Scenl 

Sea  Buoy 

470.49 

412.66 

0.22 

0.84 

Ambrose  1st  turn 

423.75 

134.29 

2.93 

0.09 

Ambrose  2nd  turn 

724.31 

195.77 

12.39 

0.006 

Verrazano  Bridge 

489.80 

57.69 

2.02 

0.18 

New  York 

Seen! 

Scenl 

Sea  Buoy 

633.41 

412.66 

0.36 

0.75 

Ambrose  1st  turn 

112.67 

134.29 

-0.05 

0.96 

Ambrose  2nd  turn 

390.82 

195.77 

0.94 

0.42 

Verrazano  Bridge 

118.61 

57.69 

0.92 

0.42 
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Figure  E-7.  Scenario  1  Composite  Trackplots 
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Figure  E-7.  Scenario  1  Composite  Trackplots  (cont.) 
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Figure  E-8.  Scenario  2  Composite  Trackplots 
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Figure  E-8.  Scenario  2  Composite  Trackplots  (cont.) 
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Figure  E-9.  Scenario  3  Composite  Trackplots 
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Figure  E-9.  Scenario  3  Composite  Trackplots  (cont.) 


Figure  E-10.  Scenario  4  Composite  Trackplot 
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ECDIS  with  Auto-Update  but  No  Radar  Overlay, 
Inbound  SF  in  Clear  Visibility  (seen.  5) 


Figure  E- 1 1 .  Scenario  5  Composite  Trackplots 
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Figure  E- 1 1 .  Scenario  5  Composite  Trackplots  (cont.) 
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Figure  E-12.  Scenario  6  Composite  Trackplots 


Figure  E- 1 3 .  Scenario  7  Composite  Trackplots 
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Figure  E-13.  Scenario  7  Composite  Trackplots  (cont.) 
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Stapleton  Anchorage 


Scenario  8 


Figure  E-14.  Scenario  8  Composite  Trackplots  (cont.) 
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Conventional  Bridge, 

Inbound  SF  in  Clear  Visibility  (seen.  9) 


Figure  E-15.  Scenario  9  Composite  Trackplots 


Figure  E-15.  Scenario  9  Composite  Trackplots  (cont.) 
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APPENDIX  F.O 

SUPPLEMENTARY  MATERIAL  FOR 
SECTION  8.0  CONCLUSIONS 


The  following  is  an  annotated  presentation  of  the  Performance  Standard 
for  ECDIS  (International  Maritime  Organization,  September  1993), 
provided  by  the  IMO/IHO  Harmonization  Group  for  ECDIS  (HGE).  In 
the  table  that  follows,  the  Standards  appear  in  the  left-hand  column,  with 
each  item  in  a  box.  To  the  right  is  a  brief  statement  of  findings  from  the 
experiment  that  are  relevant  to  that  item.  For  right-hand  statements  that 
represent  a  major  discussion  in  the  text,  the  section  number  is  included. 


Human  Factors  Evaluation  of  ECDIS 


Table  F-1.  Annotated  Presentation  of  the  Performance  Standards  for  ECDIS 


1.  INTRODUCTION 

1 . 1  The  primary  function  of  the  ECDIS  is 

to  contribute  to  safe  navigation. 

The  simulator  experiment  demonstrated  that  the 
use  of  ECDIS  can  provide  equivalent  or  greater 
safety  to  that  provided  by  the  use  of  the  paper  chart 
and  conventional  procedures  during  route 
monitoring.  Two  mechanisms  to  provide  this 
safety  were  identified:  1.)  decreased  cross  track 
distance  from  the  planned  route  and  2.)  an 
increased  proportion  of  time  spent  on  “look  out” 
and  collision  avoidance.  (Section  5.3) 

1.2  ECDIS  may  be  accepted  as  an 
equivalent  complying  with  die  up-to-date  chart 
required  by  regulation  V/20  of  the  1974 

SOLAS  Convention. 

Difficulties  with  the  accuracy  of  privately  digitized 
electronic  charts  support  the  importance  of 
hydrographic  offices  producing  or  taking 
responsibility  of  electronic  charts  if  they  are  to  be  a 
legal  equivalent.  (Section  6.1) 

1.3  In  addition  to  the  general  requirements 
for  shipbome  radio  equipment  forming  part  of 
the  global  maritime  distress  and  safety  system 
(GMDSS)  and  for  electronic  navigational  aids 
contained  in  IMO  resolution  A.694(17), 

ECDIS  should  meet  the  requirements  of  this 
performance  standard. 

1 .4  ECDIS  should  be  capable  of  displaying 
all  chart  information  necessary  for  safe  and 
efficient  navigation,  originated  by,  and 
distributed  on  the  authority  of,  government 
authorized  hydrographic  offices. 

Difficulties  with  the  accuracy  of  privately  digitized 
electronic  charts  support  the  importance  of 
hydrographic  offices  producing  or  taking 
responsibility  of  electronic  charts  if  they  are  to  be  a 
legal  equivalent.  (Section  6.1) 

1 .5  ECDIS  should  facilitate  simple  and 
reliable  updating  of  the  electronic  navigational 
chart. 

1 . 6  ECDIS  should  reduce  the  navigational 
workload  compared  to  using  the  paper  charts. 

It  should  enable  the  mariner  to  execute 
in  a  convenient  and  timely  manner  all 
route  planning,  route  monitoring  and 
positioning  currently  performed  on 
paper  charts. 

It  should  be  capable  of  continuously 
plotting  the  ship's  position. 

The  simulator  experiment  demonstrated  that  the 
use  of  ECDIS  can  reduce  the  navigation  workload 
during  route  monitoring  compared  to  using  the 
paper  chart  and  radar  and  visual  piloting 
techniques.  The  major  factor  in  the  reduction  in 
workload  is  the  automation  of  position  fixing  that 
allows  navigation  “at  a  glance.”  (Section  5.3) 

Observed  reductions  in  workload  were  dependent 
on  continuous  updating  of  the  ship’s  position. 
(Section  5.4) 
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1 .7  ECDIS  should  have  at  least  the  same 
reliability  and  availability  of  presentation  as  the 
paper  chart  published  by  government 
authorized  hydrographic  offices. 

ITS  ECDIS  should  provide  appropriate  Warnings  were  among  the  features  most  frequently 
alarms  or  indications  with  respect  to  the  recommended  by  the  participating  mariners, 

information  displayed  or  malfunction  of  the  (Section  6.4) 

equipment,  (see  appendix  5) 


T  DEFINITIONS 

2. 1  For  the  purpose  of  this  performance 
standard. 

2.1.1  Electronic  Chart  Display  Difficulties  with  the  accuracy  of  privately  digitized 
and  Information  System  (ECDIS)  means  electronic  charts  support  the  importance  of 

a  navigation  information  system  which  can  be  hydrographic  offices  producing  or  taking 
accepted  as  complying  with  the  up-to-date  chart  responsibility  of  electronic  charts  if  they  are  to  be  a 
required  by  regulation  V^O  of  the  1974  legal  equivalent.  (Section  6.1) 

SOLAS  Convention,  by  displaying  selected 

information  from  a  system  electronic  naviga-  Both  objective  data  and  mariners  opinions  strongly 
tional  chart  (SENC)  with  positional  informa-  support  the  requirement  of  automatic  position 
tion  from  navigation  sensors  to  assists  the  fixing  for  route  monitoring.  (Section  5.4) 
mariner  in  route  planning  and  route  monitor¬ 
ing,  and  if  required  display  additional  Mariners  asked  for  specific  items  and  expressed  a 

navigation-related  information.  need  for  additional,  supplemental  information 

(available  in  such  a  way  that  it  would  not  clutter 
the  screen  during  route  monitoring).  (Section  6.3, 
6.4,  8.4) 

2.1.2  Electronic  Navigational  While  using  electronic  chart  digitized  by  two 

Chart  (ENC)  means  the  database,  commercial  companies,  simulator  engineers  and 

standardized  as  to  content,  structure  and  mariners  found  enough  problems  so  that  they 

format,  issued  for  use  with  ECDIS  on  the  would  strongly  recommend  that  all  ENCs  be  done 

authority  of  government  authorizes  hydrog-  on  authority  of  national  hydrographic  offices, 
raphic  offices.  The  ENC  contains  all  the  chart  (Section  6.1) 

information  necessary  for  safe  navigation  and 

may  contain  supplementaiy  information  in  Mariners  strongly  supported  supplementary 
addition  to  that  contained  in  the  paper  chart  material  that  is  easily  available  “at  user’s  option.” 
(e.g.  sailing  directions)  which  may  be  (Section  6.3) 

considered  necessary  for  safe  navigation. 
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2J3  System  Electronic  Mariners  strongly  supported  supplementary 

Navigational  Chart  (SENC)  means  a  data  material  that  is  easily  available  “at  user’s  option.’’ 
base  resulting  from  the  transformation  of  the  (Section  6.3) 

ENC  by  ECDIS  for  appropriate  use,  updates 
the  ENC  by  appropriate  means,  and  other  data 
added  by  the  mariner.  It  is  this  data  base  that  is 
actually  accessed  by  the  ECDIS  for  display 
generation  and  other  navigational  functions  and 
is  equivalent  to  an  up-to-date  paper  chart.  The 
SENC  may  also  contain  information  from  other 
sources. 

TTA  Standard  Display  means  the  The  list  of  features  most  frequently  used  or 
level  of  ENC  information  that  should  be  shown  recommended  by  the  mariners  is  included  in  the 
when  a  chart  is  first  displayed  on  the  ECDIS.  Standard  Display.  (Section  6.3,  8.4) 

Depending  upon  the  needs  of  the  mariner,  the 

level  of  the  information  it  provides  for  route  Mariners  strongly  supported  the  capability  to 
planning  or  route  monitoring  may  be  modified  modify  the  presentation  at  user’s  option, 
by  the  mariner.  (Section  6.3,  8.4) 

Several  mariners  mentioned  a  need  for  a  “panic 
button”  that  would  return  the  display  to  something 
immediately  usable. 

2.1.5  Display  Base  means  the  level 
of  SENC  information  which  cannot  be 
removed  from  the  display,  consisting  of 
information  which  is  required  at  all  times,  in  all 
geographic  areas,  and  all  circumstance.  It  is 
not  intended  to  be  sufficient  for  safe 
navigation. 

2.2  Further  information  on  definitions  may 
be  found  in  IHO  Special  Publication  S-52, 
appendix  3  (see  appendix  1). 


3.  DISPLAY  OF  SENC 
INFORMATION 

'3TI  The  ECDIS  should  be  capable  of  dis- 

playing  all  SENC  information. 
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3.2  SENC  information  available  for  display  Mariners  preferred  a  simple  display  for  route 
during  route  planning  and  route  monitoring  monitoring,  one  that  outlined  s^e  water.  They 

should  be  subdivided  into  the  following  three  strongly  supported  the  capability  to  easily 

categories:  reference  additional  information. 

(Section  6.3,  8.4) 

Display  Base 

Standard  Display  Experimenters’  tallies  and  extensive  questionnaires 

All  Other  Information  (see  appendix  2)  reporting  mariners  use  and  recommendations  for 

specific  chart  features  to  be  displayed  for  route 
monitoring  are  presented  in  the  main  report. 
(Section  6.3,  8.4) 

"33  The  ECDIS  should  present  the  Stan-  Several  mariners  mentioned  a  need  for  a  “panic 

dard  Display  at  any  time  by  a  single  operator  button”  that  would  return  the  display  to  something 

action.  immediately  usable. 

A  concern  for  standardization  of  systems  also 

_ supports  a  “standard”  display. _ 

3.4  When  a  chart  is  first  displayed  on  the  In  the  Harbor/Harbor  Approach  phase  of 
ECDIS,  it  should  provide  the  Standard  Display  navigation,  mariners  most  firequently  selected 
at  the  largest  scale  available  in  the  SENC  for  scales  in  the  1:20,000  to  1:50,000  range, 
the  displayed  area.  Occasionally,  they  zoomed  to  larger  scales. 

(Section  6.2) 

"33  It  should  be  easy  to  add  or  remove  The  mariners  tended  to  prefer  a  relatively  simple 

information  from  the  ECDIS  display.  It  should  display  for  route  monitoring.  Also,  they  were 

not  be  possible  to  remove  information  generdly  in  favor  of  “at  user’s  option”  in  the 

contained  in  the  Display  Base.  selection  of  features.  (Section  6.3,  8.4) 

3.6  It  should  be  possible  for  the  mariner  to 
select  a  safety  contour  from  the  depth  contours 
provided  by  the  SENC.  ECDIS  should 
emphasize  the  safety  contour  over  other 
contours  on  the  display. 

3.7  It  should  be  possible  for  the  mariner  to  How  a  safety  depth  should  be  displayed  was  a 

select  a  safety  depth.  The  ECDIS  should  major  concern.  Several  recommended  that 

emphasize  soundings  equal  to  or  less  than  the  soundings  or  contours  be  expressed  in  terms  of 

safety  depth  whenever  spot  soundings  are  underkeel  clearance  for  own  ship,  rather  than 

selected  for  display.  absolute  depth.  One  wanted  underkeel  clearance 

calculated  for  speed  of  own  ship.  (Section  6.3) 


3 . 8  The  ENC  and  all  update  information 
should  be  displayed  without  degradation  of 
their  information  content. 

3.9  The  system  should  provide  a  method  to 
ensure  the  ENC  and  all  updates  to  it  have  been 
correctly  loaded  into  the  SENC. 
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3.10  The  ENC  data  and  updates  to  it  should 
be  clearly  distinguishable  from  other  displayed 
information,  such  as,  for  example,  that  lists  in 
appendix  3. 


4 .  PROVISION  AND  UPDATING 
OF  CHART  INFORMATION  _ 


4. 1  The  chart  information  to  be  used  m 
ECDIS  should  be  the  latest  edition  of  that  origi¬ 
nated  by  a  government  authorized 
hydrographic  office,  and  conform  to  IHO 
standards. 


4.2  The  contents  of  the  SENC  should  be 

adequate  and  up-to-date  for  the  intended 
voyage  to  comply  with  regulation  V/20  of  the 
1974  SOLAS  Convention. 


It  should  not  be  possible  to  alter  the 
contents  of  the  ENC. 


4.4  Updates  should  be  stored  separately 
from  the  ENC. 


The  ECDIS  should  be  capable  of 
accepting  official  updates  to  the  ENC  data 
provided  in  standard  IHO  format.  These  up¬ 
dates  should  be  automatically  applied  to  the 
SENC.  By  whatever  means  updates  are 
received,  the  implementation  procedure  should 
not  interfere  with  the  display  in  use. 


The  ECDIS  should  also  be  capable  of 
accepting  updates  to  the  ENC  data  entered 
manually  with  simple  means  for  verification 
prior  to  the  final  acceptance  of  the  data.  They 
should  be  distinguishable  on  the  display  from 
ENC  information  and  its  official  updates  and 
not  affect  its  legibility. 


4.7  ECDIS  should  keep  a  record  of  updates 
including  time  of  application  to  the  SENC. 


4. 8  ECDIS  should  allow  the  manner  to 
display  updates  in  order  to  view  their  contents 
and  to  ascertain  that  they  have  been  included  in 
the  SENC. 
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5.  SCALE 

The  ECDIS  should  provide  an 
indication  if: 

.  1  the  information  is  displayed  at  a  larger 

scale  than  that  contained  in  the  ENC;  or 

,2  own  ship's  position  is  covered  by  an 

ENC  at  a  larger  scale  than  that  provided 
by  the  display,  a  warning  should  be 
provided. 

Mariners  did  not  appear  to  understand  the 
implications  of  scale.  This  is  an  matter  for  training 
as  well  as  warnings. 

6.  DISPLAY  OF  OTHER 
INFORMATION 

6. 1  Radar  information  or  other  navigational 

information  may  be  added  to  the  ECDIS 
display.  However,  it  should  not  degrade  the 
SENC  information  and  it  should  be  clearly 
distinguishable  from  the  SENC  information. 

Mariners’  concerns  with  the  clarity  and  clutter  of 
the  integrated  displays  that  they  saw  support  this 
item.  (Section  7.2) 

6.2  ECDIS  and  added  navigational 
information  should  use  a  common  reference 
system.  If  this  is  not  the  case,  an  indication 
should  be  provided. 

6.3  Radar 

6.3.1  Transferred  radar  information 
may  contain  both  the  radar  image  and  ARPA 
information. 

Mariners  wanted  a  complete,  high  fidelity 
radar/ARPA  both  for  navigation  and  for  collision 
avoidance.  This  need  makes  integration  a  complex 
issue  beyond  the  scope  of  this  Standard. 

(Section  7.2) 

6.3.2  If  the  radar  image  is  added  to 
the  ECDIS  display,  the  chan  and  the  radar 
image  should  match  in  scale  and  in  orientation. 

6.3.3  The  radar  image  and  the 
position  from  the  position  sensor  should  both 
be  adjusted  automatically  for  antenna  offset 
from  the  conning  position. 

The  match  between  radar  and  chart  image  was  an 
important  factor  in  mariner  confidence.  Some  saw 
it  as  the  primaiy  value  of  a  radar  overlay. 

(Section  7.2) 

6.3.4  It  should  be  possible  to  adjust 
the  displayed  position  of  the  ship  manually  so 
that  the  radar  image  matches  the  SENC  dis¬ 
play. 
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03  It  should  be  possible  to  remove  I  In  questionnaires,  mariners  recommended 
the  radar  information  by  single  operator  action,  flexibility  and  user’s  option  in  how  the  integration 

was  presented.  (Section  7.2) 


T.  DISPLAY  MODE  AND 
GENERATION  OF  THE 
NEIGHBORING  AREA 

71  It  should  always  be  jwssible  to  display  Of  the  mariners  who  expressed  a  preference,  it 
the  SENC  in  a  "north-up"  orientation.  Other  was  for  north-up. 
orientations  are  permitted. 

7^7  ECDIS  should  provide  for  true  motion  Mariners  wanted  t»th  true  and  relative  motion, 
mode.  Other  modes  are  permitted.  especially  for  use  in  traffic. 

73  When  true  motion  mode  is  in  use,  reset  Mariners  supported  ^y  features  that  resulted  in 
and  generation  of  the  neighboring  area  should  “hands  off’  navigation, 
take  place  automatically  at  a  distance  from  the 
border  of  the  display  determined  by  the  mari¬ 
ner. 

71  It  should  be  possible  to  change  manu-  Mariners  felt  that  this  was  an  important  feature  for 
ally  the  chart  area  and  the  position  of  own  ship  “look  ahead.” 
relative  to  the  edge  of  the  display. 


8.  COLOURS  AND  SYMBOLS 

^1  MO  recommended  colours  and  sym- 
bols  should  be  used  to  represent  ENC  infor¬ 
mation.  (MO  Special  Publication  S-52, 
appendix  2)  (see  appendix  1) 

"83  The  colours  and  symbols  other  than 
those  mentioned  in  8.1  should  be  those  used  to 
describe  the  navigational  elements  and 
parameters  listed  in  appendix  3  and  published 
by  lEC  Publication  1 174. 

"O  SENC  information  when  displayed  at 
the  scale  specified  in  the  ENC  should  use  the 
specified  size  of  symbols,  figures  and  letters. 

(MO  Special  Publication  S-52,  appendix  2  - 
see  appendix  1  and  lEC  Publication  1174) 

"83  The  ECDIS  should  allow  the  mariner  to  After  seeing  both  scaled  and  unsealed  ship’ 
select  whether  own  ship  is  displayed  in  true  symbols  in  Harbor/Harbor  Approach  operations, 
scale  or  as  a  symbol.  mariners  strongly  preferred  a  scaled  symbol. 

(Section  6.4) 
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9 .  DISPLAY  REQUIREMENTS 

9. 1  ECDIS  should  be  capable  of  displaying 

information  for: 

.  1  route  planning  and  supplementary 
navigation  tasks 
.2  route  monitoring 

The  findings  of  the  experiment  supported  the 
substantial  value  of  ECDIS  for  route  monitoring. 
(Section  5.0) 

Both  a  second  chart  view  and  the  capability  to 
offset  the  image  of  own  ship  were  valued  for  look 
ahead. 

9.2  The  effective  size  of  the  chart 
presentation  for  route  monitoring  should  be  at 
least  270  mm  by  270  mm. 

The  mariners  were  satisfied  with  this  chart  size. 

But  they  saw  it  with  the  availability  of  a  second 
chart  presentation  set  at  a  smaller  scale  for  look 
ahead  or  set  at  a  larger  scale  for  greater  resolution. 
(Section  6.2) 

9.3  The  display  should  be  capable  of 

meeting  colour  and  resolution 
recommendations  of  MO. 

9.4  The  method  of  presentation  should 

ensure  that  the  displayeii  information  is  clearly 
visible  to  more  than  one  observer  in  the  condi¬ 
tions  of  light  normally  experienced  on  the 
bridge  of  the  ship  by  day  and  by  night. 

10.  ROUTE  PLANNING, 
MONITORING  AND  VOYAGE 
RECORDING 

10. 1  It  should  be  possible  to  carry  out  route 
planning  and  route  monitoring  in  a  simple  and 
reliable  manner. 

The  study  considered  only  route  monitoring. 
Performance  is  described  in  Section  5.3  ,5.4, 
Features  and  functions  used  or  recommended  are 
described  in  Section  6.3,  6.4. 

10.2  ECDIS  should  be  designed  following 
ergonomic  principles  for  user-friendly  opera¬ 
tion. 

The  simulator  experiment  examined  the  mariner’s 
preferences  for  system  features  to  be  used  during 
route  monitoring.  (Section  6.3)  It  did  not 
examine  more  generic  operator/computer  interface 
issues.  (Section  8.7) 

10.3  The  largest  scale  data  available  in  the 
SENC  for  the  area  given,  shall  always  be  used 
by  the  ECDIS  for  all  alarms  and  indications  of 
crossing  the  ship's  safety  contour  and  of 
entering  a  prohibited  area,  and  for  alarms  and 
indications  according  to  Appendix  4. 
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TO  Route  Flanning 

10.4.1  It  should  be  possible  to  carry 
out  route  planning  including  both  straight  and 
ciuved  segments. 

10.4.2  It  should  be  possible  to  adjust 
a  planned  route  by,  for  example: 

.  1  Ading  waypoints  to  a  route; 

.2  Deleting  waypoints  from  a  route; 

.3  Changing  the  position  of  a 
waypoint; 

.4  Changing  the  order  of  the 
waypoints  in  the  route. 

10.4.3  It  should  be  possible  to  plan 
alternate  routes  in  addition  to  the  selected  route. 
The  selected  route  should  be  clearly  disting¬ 
uishable  from  the  other  route. 

10.4.4  An  indication  is  required  if  the 
mariner  plans  a  route  across  an  own  ship's 
safety  contour. 

10.4.5  An  indication  is  required  if  the 
mariner  plans  a  route  across  the  boundary  of  a 
prohibited  area  or  a  geographic  area  for  which 
special  conditions  exist.  (See  Appendix  4) 

10.4.6  It  should  be  possible  for  the 
mariner  to  specify  a  limit  of  deviation  from  the 
planned  route  at  which  activation  of  the 
automatic  offtrack  alarm  should  occur. 

10.5  Route  Monitoring 

10.5.1  For  route  monitoring  the  se- 
lected  route  and  own  ship's  position  should  ap¬ 
pear  whenever  the  display  covers  that  area. 
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10.5.2  It  should  be  possible  to  dis-  Both  a  second  chart  view  and  the  capability  to 
play  a  sea  area  that  does  not  have  the  ship  on  offset  the  image  of  own  ship  were  valued  for  look 
the  display  (e.g.  for  look  ahead,  route  plan-  ahead, 
ning)  while  route  monitoring. 

Warnings  were  among  the  features  most  frequently 
If  this  is  done  on  the  display  used  recommended  by  the  participating  mariners, 
for  route  monitoring  functions  (Section  6.4) 

(e.g.  updating  ship’s  position, 

and  providing  alarms  and  Several  mariners  mentioned  a  need  for  a  “panic 

indications)  should  be  button”  that  would  return  the  display  to  something 

continuous.  immediately  usable. 


the  route  monitoring  display  cov¬ 
ering  own  ship's  position 
immediately  by  single  operator 
action. 

10.5.3  ECDIS  should  give  an  alarm 
if ,  within  a  specified  time  set  by  the  mariner, 
own  ship  will  cross  the  safety  contour. 

10.5.4  ECDIS  should  give  an  alarm 
if,  within  a  specified  time  set  by  the  mariner, 
own  ship  will  cross  the  boundary  of  a 
prohibited  area  or  a  geographical  area  for 
which  special  conditions  exist.  (See  Appendix 
4.) 

10.5.5  An  alarm  should  be  given 
when  the  specified  limit  for  deviation  from  the 
planned  route  is  exceeded. 

10.5.6  The  ship's  position  should  be  The  major  factor  in  the  reduction  in  workload  is 
derived  from  a  continuous  positioning  system  the  automation  of  position  fixing  that  allows 

of  an  accuracy  consistent  with  the  requirements  navigation  “at  a  glance.”  (Section  5.3,  5.4) 
of  safe  navigation.  A  second  independent 

positioning  method  of  a  different  type  should  Mariners  were  extremely  concerned  with  the 
be  provided;  ECDIS  should  be  capable  of  accuracy  of  all  aspects  of  the  display, 
identifying  discrepancies  between  the  two  (Section  6.1) 

systems. 

10.5.7  ECDIS  should  provide  an  Warnings  were  among  the  features  most  frequently 
indication  when  the  input  from  the  position  recommended  by  the  participating  mariners, 
fixing  system  is  lost.  ECDIS  should  also  (Section  6.4) 

repeat,  but  only  as  an  indication,  any  alarm  or 
indication  passed  to  it  from  a  positioning  fixing 
system. 

10.5.8  An  alarm  should  be  given  by 
ECDIS  if  the  ship,  within  a  specified  time  or 
distance  set  by  the  mariner,  will  reach  a  critical 
point  on  the  planned  route. 
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10.5.9  The  positioning  system  and 
the  SENC  should  be  on  the  same  geodetic 
datum.  ECDIS  should  give  an  alarm  if  this  is 
not  the  case. 

10.5. 10  It  should  be  possible  to  dis¬ 
play  alternate  routes  in  addition  to  the  selected 
route. 

The  selected  route  should  be 
clearly  distinguishable  from 
the  other  routes. 

During  the  voyage,  it  should 
be  possible  for  the  mariner  to 
modify  the  selected  sailing 
route  or  change  to  an 
alternative  route. 

10.5. 11  It  should  be  possible  to 

display: 

,  1  time  labels  along  ships  track 
manually  on  demand  and  auto¬ 
matically  at  intervals  selected 
between  1  and  120  minutes;  and 

.2  an  adequate  number  of:  points, 
free  movable  electronic  bearing 
lines,  variable  and  fixed  range 
markers,  and  other  symbols 
required  for  navigation  purposes 
as  specified  in  Appendix  3. 

A  frequently  requested  feature  was  Estimate  Time 
of  Arrival  (ETA)  to  a  point  indicated  with  the 
cursor. 

Most  mariners  wanted  the  option  of  displaying  or 
not  displaying  past  track.  Past  track  data  on  the 
display  caused  a  problem  when  a  run  was  repeated 
without  turning  off  the  system.  The  capability  to 
immediately  remove  (and/or  store)  a  past  track 
would  be  important  to  ferry  operations. 

10.5.12  It  should  be  possible  to  enter 
the  geographical  coordinates  of  any  position 
and  then  display  that  position  on  demand. 

Also,  it  should  be  possible  to 
select  any  point  (features, 
symbol  or  position)  on  the 
display  and  read  it’s 
geographical  coordinates  on 
demand. 

Mariners  were  not  concerned  with  lat/long  position 
except  when  it  was  needed  to  communicate  to 
another  ship. 

10.5. 13  It  should  be  possible  to 
adjust  the  ship's  geographic  position  manually. 

This  manual  adjustment 
should  be  noted 
alpha-numerically  on  the 
screen,  maintain^  until 
altered  by  the  mariner  and 
automatically  recorded. 
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10.6  Voyage  Recording 

10.6.1  ECDIS  should  store  and  re¬ 
produce  certain  minimum  elements  required  to 
reconstruct  the  navigation  and  verify  the 
official  database  used  during  the  previous  12 
hours.  The  following  data  shall  be  recorded  at 

1  minute  intervals: 

.  1  to  ensure  a  record  of  own  ship's  past 
track:  time,  position,  heading,  and 
speed;  and 

.2  to  ensure  a  record  of  official  data 
used:  ENC  source,  edition,  date,  cell 
and  update  history. 

10.6.2  In  addition,  ECDIS  should 
record  the  complete  track  for  the  entire  voyage, 
with  time  marks  at  intervals  not  exceeding  4 
hours. 

10.6.3  ECDIS  should  have  a 
capability  to  preserve  the  record  of  the  previous 
12  hours  and  the  voyage  track. 

11.  ACCURACY 

11.1  The  accuracy  of  all  calculations 
performed  by  ECDIS  should  be  independent  of 
the  characteristics  of  the  output  device  and 
should  be  consistent  with  SENC  accuracy. 

11.2  Bearings  and  distances  drawn  on  the 
display  or  those  measured  between  features 
already  drawn  on  the  display  should  have  an 
accuracy  no  less  than  that  afforded  by  the 
resolution  of  the  display. 

'I 


F-12 


IMO  NAV39/WP.2/Add.2/Sep93 


Human  Factors  Evaluation  of  ECDIS 


12.  CONNE 
EQUIPMENT 


OTHER 


12.1  The  ECDIS  should  not  degrade  the  per¬ 
formance  of  any  equipment  providing  sensor 
inputs.  Nor  should  the  connection  of  optional 
equipment  degrade  the  performance  of  ECDIS 
below  this  standard.  _ 


12.2  ECDIS  should  be  connected  to  systems 
providing  continuous  position  fixing,  heading 
and  speed  information. 


13.  MALFUNCTION  WARNINGS 
AND  PERFORMANCE  TESTS 


13.1  ECDIS  should  be  provided  with  means 
for  either  automatically  or  manually  carrying 
out  on  board  tests  of  major  functions.  In  case 
of  a  failure,  the  test  should  display  information 
to  indicate  which  module  is  at  fault. 


13.2  ECDI S  should  provide  suitable  alarm 

or  indication  of  system  malfunction. 


Warnings  were  among  the  features  most  frequently 
recommended  by  the  participating  mariners. 
(Section  6.4) 


:k-up  arrangements 


14.1  Adequate  back-up  arrangements  should 
be  provided  to  ensure  safe  navigation  in  case  of 
ECDIS  failure. 

.  1  Facilities  enabling  a  safe  take-over 
of  the  ECDIS  functions  should  be 
provided  to  avoid  that  an  ECDIS 
failure  develops  into  a  critical  situa¬ 
tion. 

.  2  A  back-up  arrangement  facilitating 
means  for  safe  navigation  of  the 
remaining  part  of  the  voyage  in  case 
of  ECDIS  failure  should  be 
provided. 


IMO  NAV39/WP.2/Add.2/Sep93 


Human  Factors  Evaluation  of  ECDIS 


15.  POWER  SUPPLY 

15.1  It  should  be  possible  to  operate  the 
ECDIS  and  all  equipment  necessary  for  its 
normal  functioning  when  supplied  by  an 
emergency  source  of  electrical  power  in  ac¬ 
cordance  with  the  appropriate  requirements  of 
Chapter  n-1  of  the  1974  SOLAS  Convention. 

15.2  Changing  from  one  source  of  power 
supply  to  another  or  any  interruption  of  the 
supply  for  a  period  of  up  to  45  seconds  should 
not  require  the  equipment  to  be  manually 
re-initialized 
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